Traceback (most recent call last):
File "train.py", line 151, in
train()
File "train.py", line 102, in train
train_dataset = DeepfakeDataset('train', cfg)
File "/home/lab/tSun/CADDM-master/dataset.py", line 37, in init
self.samples = self.collect_samples()
File "/home/lab/tSun/CADDM-master/dataset.py", line 56, in collect_samples
info_meta = self.info_meta_dict[info_key]
KeyError: './train_images/manipulated_sequences/Deepfakes/raw/frames/585_599/frame_0'

I ran into this problem while running the program. Can you help me solve it!

Thank you for the new insight in deepfake detection!
I have some questions for fully understanding the "Implicit Identity Leakage".
The FF++ real dataset has 1000 videos, and the author split them into 720, 140, 140 for train, val and test.
Taking the training set as an example, the fake videos used for training are generated by forging each other within the training set.
Therefore, on FF++, the face identity information of the training set and the test set do not overlap.
So if the network has learned a lot of features related to face identity, the in-dataset test AUC on FF++ dataset will also lead to a decrease because these face identities have not been seen during training.

I would like to know how the data for the experimental training is set up, how many frames are taken from one video? For example, how many frames are taken for each of the 5 methods?(youtube, Deepfakes, FaceSwap...) As shown in the code of extract_frames_ldm_ff++.py, is it only one frame taken from each video? Thanks for your reply and work!

I run python test.py --cfg ./configs/caddm_test.cfg as is indicated in your readme, and I expect a successful completion of evaluation task. Instead, I got an error:

(caddm) molokanov@molokanov-pc:~/CADDM$ python test.py --cfg ./configs/caddm_test.cfg
Traceback (most recent call last):
  File "test.py", line 83, in <module>
    test()
  File "test.py", line 41, in test
    cfg = load_config(args.cfg)
  File "/home/molokanov/CADDM/lib/util.py", line 10, in load_config
    config = yaml.load(f)
TypeError: load() missing 1 required positional argument: 'Loader'

My conda env is as follows:

dlib              19.24.2    
imutils           0.5.4      
joblib            1.3.2      
numpy             1.19.5     
opencv-python     4.9.0.80   
pillow            10.2.0     
pip               20.0.2     
pkg-resources     0.0.0      
scikit-learn      1.3.2      
scipy             1.10.1     
setuptools        44.0.0     
threadpoolctl     3.3.0      
torch             1.8.1+cu101
torchaudio        0.8.1      
torchvision       0.9.1+cu101
tqdm              4.66.2     
typing-extensions 4.9.0

Can you help me?

sorry I misread some settings myself

Dear Esteemed Researchers,
The first question is that when I extract frames from videos, there exists some faces in the frames can't be detected so that we can't record the landmarks. Sometimes the source frame can't be detected so that it won't exist in the train or test images, but it will be recorded into .json with its target image. The dataloader will return an error 'None'. So I used to try to delete all the keys in which the source frame is not extracted. But then the dataset will just remain almost a half data. How do you solve this problem? The second question is that how do you solve the fairness when trying to compare with other methods. I think the data in your method will be less than other methods.

Dear Esteemed Researchers,
Thank you very much for your research results. Is it convenient to share the checkpoints.

Hello there,
I am reaching out to seek your guidance regarding the implementation phase. Should we begin by training the model using the dataset provided, or would it be advisable to utilize the pretrained weights and model made available through the provided link? (https://drive.google.com/file/d/1JNMI4RGssgCOl9t05jkUa6imnw5XR5id/view?usp=sharing)
Thank you for your time and consideration.

@zsc @bigeagle @longern @dc3671 @Comedi
I am trying to reproduce test results for celeb-DF, but the accuracy is just 86.06. Maybe I made some mistakes while I was generating the dataset. Can you please provide the code to produce a test dataset using Celeb-DF? Some files like 'lib/extract_frames_ldm_ff++.py' for celeb-DF.

Thanks.

The .so file was generated after running 'make', but importing _jpegpy failed.

(pt) darui@darui-PC:~/CODE/DFD/CADDM-master/lib/data_preprocess/utils/jpegpy$ make
g++ _jpegpy.cc -o _jpegpy.so -std=c++11 -O3 -I/home/darui/APP/anaconda3/envs/pt/include/python3.9 -I/home/darui/APP/anaconda3/envs/pt/include/python3.9 -Wno-unused-result -Wsign-compare -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -ffunction-sections -pipe -isystem /home/darui/APP/anaconda3/envs/pt/include -fdebug-prefix-map=/croot/python-split_1688589050410/work=/usr/local/src/conda/python-3.9.17 -fdebug-prefix-map=/home/darui/APP/anaconda3/envs/pt=/usr/local/src/conda-prefix -I/home/darui/APP/anaconda3/envs/pt/include -DNDEBUG -O3 -Wall -UNDEBUG -L/build/libjpeg-turbo-Iy19nI/libjpeg-turbo-2.0.3/obj-x86_64-linux-gnu/lib/x86_64-linux-gnu -ljpeg -L/home/darui/APP/anaconda3/envs/pt/lib/python3.9/config-3.9-x86_64-linux-gnu -L/home/darui/APP/anaconda3/envs/pt/lib -lcrypt -lpthread -ldl -lutil -lm -lm -shared -fPIC
In file included from pybind11/pytypes.h:12,
from pybind11/cast.h:13,
from pybind11/attr.h:13,
from pybind11/pybind11.h:36,
from _jpegpy.cc:6:

When generating a json file with my own data set,

height, width = frame.shape[:-1] # Gets the height and width of the image

AttributeError: 'NoneType' object has no attribute 'shape'. In fact, the size of the video that cannot get shape is the same as the size of the video that does not report error, which is 720✖1280

Hi, I tried to run test.py and met an error "cannot import name '_jpegpy'". It seems that _jpegpy.cc can not be imported straightforwardly in python.

hi, can you tell the performance on the FFIW DFDCP dataset?

It seems that in the process of frame extraction, there will be missing frames due to the inability to detect faces, thus reducing the size of the dataset, will this affect the training effect, and if so, how to solve this problem？

Hi, I am trying to reproduce your training results and have a question about the batch size.
From the paper, I see

During the training phase, we set the batch size to 128 ...
We set the number of total epochs to 200, each of which had 512 randomly selected mini-batches.

After running the preprocessing, I get total around 32 * 720 * 5 = 115200 images.
By mini-batches, do you mean you used random 512 batches per epoch, each size of 128?

Hi，I have a question.can you help me, please?

hi，I got some issue. Python 3.10.6. When I imported _jpegpy, i got an error

Traceback (most recent call last):
  File "/home/ap/jpegpy/jpegpy.py", line 6, in <module>
    import _jpegpy
ImportError: /home/ap/jpegpy/_jpegpy.so: undefined symbol: `jpeg_mem_src``

For the make，i can compile the generated so file, but there are warnings

make: pkg-config: No such file or directory
make: mdl: No such file or directory
make: pkg-config: No such file or directory
make: mdl: No such file or directory
g++ _jpegpy.cc -o _jpegpy.so -std=c++11 -O3 -UNDEBUG --shared -fPIC
In file included from pybind11/pytypes.h:12,
from pybind11/cast.h:13,
from pybind11/attr.h:13,
from pybind11/pybind11.h:36,
from _jpegpy.cc:6:
pybind11/common.h:393:33: warning: ‘int PyThread_create_key()’ is deprecated [-Wdeprecated-declarations]
393 | decltype(PyThread_create_key()) tstate = 0; // Usually an int but a long on Cygwin64 with Python 3.x
| ~~~~~~~~~~~~~~~~~~~^~
In file included from /usr/include/python3.10/Python.h:122,
from pybind11/common.h:99,
from pybind11/pytypes.h:12,
from pybind11/cast.h:13,
from pybind11/attr.h:13,
from pybind11/pybind11.h:36,
from _jpegpy.cc:6:
/usr/include/python3.10/pythread.h:108:36: note: declared here
108 | Py_DEPRECATED(3.7) PyAPI_FUNC(int) PyThread_create_key(void);
| ^~~~~~~~~~~~~~~~~~~

My supervisor have done the procedure to receive the downloading script from the FF++ authors. However, upon running the script, my device failed to gain connection with the download server. I tried accessing the server link directly and it shows "site cannot be reached. What should I do to continue? Thank you.

Dear Authors,

I hope this finds you well. I am reaching out to you with regard to a matter of keen interest to me. I am currently working on testing the code and model weights provided in a recent piece of work on deepfake detection. However, I am encountering some challenges while trying to download the needed datasets, specifically the DFDC and Celeb-DF datasets.

Firstly, as I understand it, downloading the DFDC dataset requires registration through an AWS account. Despite multiple attempts, I have not been successful in completing this registration. I am unsure if there is an issue with the process or if the problem lies on my end. I did, however, stumble upon a similar dataset on Kaggle, the details of which can be found at this link: Kaggle Deepfake Detection Challenge Dataset. Could you please confirm if this is the same dataset used in the work I am referring to? If not, could you guide me on how to obtain a download to the DFDC dataset?

Secondly, while trying to download the Celeb-DF dataset, I found two versions available - Celeb-DF v1 and Celeb-DF v2. Could you kindly clarify which version was used in the paper?

I would greatly appreciate any assistance or guidance you can provide on these matters. I believe your insights will be invaluable in advancing my work in this important area.

Looking forward to your response.

Best Regards,
JC

如果设置extarct frame比较小的话，extract_frames_ldm_ff++.py提取的ldm.json文件中许多source_img在提取出的图片中并不存在

How to test on different datasets? (i.e. Celeb-DF, DFDC)

Now there is only extract_frames_ldm_ff++.py in the project.

Will the extract method of Celeb-DF and DFDC be released in the future?

Has anyone successfully tested it on other datasets?

Hello, thanks for sharing your codes.

when I run make in CADDM/lib/data_preprocess/utils/jpegpy, It produces;

lto1: internal compiler error: in lto_tag_to_tree_code, at lto-streamer.h:1005
Please submit a full bug report,
with preprocessed source if appropriate.
See file:///usr/share/doc/gcc-7/README.Bugs for instructions.
lto-wrapper: fatal error: g++ returned 1 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
Makefile:10: recipe for target '_jpegpy.so' failed
make: *** [_jpegpy.so] Error 1

I checked other issues related to _jpegpy.so and tried all of them, but they did not work.
Thank you in advance.

Hello, can your code run under Windows? I have successfully run your code under Ubuntu. But under windows I always get the error “ImportError: cannot import name '_jpegpy' from partially initialized module 'lib.data_preprocess.utils.jpegpy' (most likely due to a circular import)”, this problem has been solved under Ubuntu by generating .so files, I would like to bother to ask why

I have a question，SBI AUC score is 93.18 in Celeb-DF dataset，Have you had a successful reproduction?I ran SBI official train code，only got 91.3。

Dear authors,
I thank you a lot for your contribution to this field and for providing the resource for us, I am new to this field and after encountering some problems that I resolved based on the issues issued here, I got to the testing part; but when I run the command you specified in the readme; I got the following error
Traceback (most recent call last):
File "/media/workstation/DATA/CADDM-master/test.py", line 15, in
from dataset import DeepfakeDataset
File "/media/workstation/DATA/CADDM-master/dataset.py", line 10, in
from lib.data_preprocess.preprocess import prepare_train_input, prepare_test_input
File "/media/workstation/DATA/CADDM-master/lib/data_preprocess/preprocess.py", line 14, in
from .mfs import multi_scale_facial_swap
File "/media/workstation/DATA/CADDM-master/lib/data_preprocess/mfs.py", line 9, in
from .augmentor import elastic_transform
File "/media/workstation/DATA/CADDM-master/lib/data_preprocess/augmentor.py", line 12, in
from .utils.jpegpy import jpeg_decode, jpeg_encode
File "/media/workstation/DATA/CADDM-master/lib/data_preprocess/utils/jpegpy/init.py", line 6, in
from .jpegpy import jpeg_encode, jpeg_decode
File "/media/workstation/DATA/CADDM-master/lib/data_preprocess/utils/jpegpy/jpegpy.py", line 6, in
from . import _jpegpy
ImportError: /media/workstation/DATA/CADDM-master/lib/data_preprocess/utils/jpegpy/_jpegpy.so: undefined symbol: jpeg_destroy_decompress

I am unable to resolve the issue, and quite surprised since I followed the same steps as people before me; I would appreciate your help a lot.
Thank you.

Thank you very much for your contributions in the field of deep forgery detection. As a newbie, I wondered how to compile in the jpegpy folder. Also, I can't always find python.h, even though I have devel installed!

I got the following error when running test.py:
Load deepfake dataset from ./test_images..
Traceback (most recent call last):
File "E:\mask\CADDM-master\CADDM-master\test.py", line 91, in
test()
File "E:\mask\CADDM-master\CADDM-master\test.py", line 62, in test
test_dataset = DeepfakeDataset('test', cfg)
File "E:\mask\CADDM-master\CADDM-master\dataset.py", line 37, in init
self.samples = self.collect_samples()
File "E:\mask\CADDM-master\CADDM-master\dataset.py", line 56, in collect_samples
info_meta = self.info_meta_dict[info_key]
KeyError: './test_images\manipulated_sequences\NeuralTextures\c23\frames\000_003\frame_0'

进程已结束,退出代码1

What is the question, please

Hello, thank you for sharing the code. I would like to ask, after retraining for 80 epochs, the AUC of the model on the FF++dataset is around 0.98, but the AUC across datasets on the CELEB-DF-V2 dataset is only around 0.74, which is very poor. May I ask why this is the case?

Thank you for the new insight in deepfake detection!
I have some questions for running the training code. When compiling the Makefile file, it shows that there is no jpeglib.h file

My F ++ data set has been converted to a picture, how should I modify the code? When modifying the generated json file, is the json file used for training?

The pre-trained model you gave only has a .pkl file but not a .pt file. Where can I get this file?

Thanks to the author for his contribution to deepfakes detection.
As mentioned in your paper, the method of this paper can be combined with the SBI method on paper.
I don’t know if you can release the code or method combined with SBI？

Thank you !

I used rtx3080ti cuda11.1, python==3.8, and the others were configured according to the environment given by the author, but I encountered problems during training

Load deepfake dataset from ./train_images..
Target RGB shape: ()
Target RGB shape: (480, 712, 3)
Target RGB shape: ()
Traceback (most recent call last):
File "train.py", line 151, in
train()
File "train.py", line 112, in train
for index, (batch_data, batch_labels) in enumerate(train_loader):
File "/home/lab/anaconda3/envs/DeepfakeBench/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 517, in next
data = self._next_data()
File "/home/lab/anaconda3/envs/DeepfakeBench/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1199, in _next_data
return self._process_data(data)
File "/home/lab/anaconda3/envs/DeepfakeBench/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1225, in _process_data
data.reraise()
File "/home/lab/anaconda3/envs/DeepfakeBench/lib/python3.8/site-packages/torch/_utils.py", line 429, in reraise
raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/lab/anaconda3/envs/DeepfakeBench/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 202, in _worker_loop
data = fetcher.fetch(index)
File "/home/lab/anaconda3/envs/DeepfakeBench/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/lab/anaconda3/envs/DeepfakeBench/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/lab/tSun/CADDM-master/dataset.py", line 81, in getitem
img, label_dict = prepare_train_input(
File "/home/lab/tSun/CADDM-master/lib/data_preprocess/preprocess.py", line 92, in prepare_train_input
mfs_result, bbox = multi_scale_facial_swap(
File "/home/lab/tSun/CADDM-master/lib/data_preprocess/mfs.py", line 130, in multi_scale_facial_swap
h, w = targetRgb.shape[:2]
ValueError: not enough values to unpack (expected 2, got 0)

I think it is caused by some images being empty when loading images, but I don't know how to solve it.

In extract_frames_ldm_ff++.py the default parameter for the variable NUM_FRAMES is 1. In this case there is no problem, in terms of corresponding frames, if only one frame is taken from each video, for a given fake video, its frame 0 will correspond perfectly with its source video frame 0. The problem comes when NUM_FRAMES is not 1. Let's consider NUM_FRAMES = 30. In this case, according to this code the frame index that will be taken will follow a linear distribution so if there are 300 frames and 30 frame are desired, frames 0,10,20,...,290,300 are taken. This is a problem since source videos and fake videos don't always have the same length so ifi 30 frames are desired for video fake A, frames 0,10,20,30,...300 are taken but if video source B has, for example 270 frames, then frames 0,9,18,27,37...270 will be taken. It is a small detail and to some extent is not even a problem because the frames will be very similar but the source paths are not entirely accurate.

After running make , with _jpegpy.so created ,it produces:
In file included from _jpegpy.cc:6:
pybind11/pybind11.h: In destructor ‘pybind11::gil_scoped_release::~gil_scoped_release()’:
pybind11/pybind11.h:1611:47: warning: ‘int PyThread_set_key_value(int, void*)’ is deprecated [-Wdeprecated-declarations]
1611 | PyThread_set_key_value(key, tstate);
| ^
In file included from /home/anaconda3/envs/general/include/python3.9/Python.h:131,
from pybind11/common.h:99,
from pybind11/pytypes.h:12,
from pybind11/cast.h:13,
from pybind11/attr.h:13,
from pybind11/pybind11.h:36,
from _jpegpy.cc:6:
/home/anaconda3/envs/general/include/python3.9/pythread.h:110:36: note: declared here
110 | Py_DEPRECATED(3.7) PyAPI_FUNC(int) PyThread_set_key_value(int key,
| ^~~~~~~~~~~~~~~~~~~~~~

But running python test.py, it produces:
File "/home/CADDM-master/lib/data_preprocess/utils/jpegpy/jpegpy.py", line 5, in <module> from . import _jpegpy ImportError: /home/CADDM-master/lib/data_preprocess/utils/jpegpy/_jpegpy.so: undefined symbol: jpeg_mem_src

Hoping to get a reply.Thanks

Thanks for this new deepfake detection work! I tested this project and find the bboxes may bring biases during generation. Does this phenomenon may harm the performance?

Hello!
I think I found a bug in the poisson_blending_func function. It seems that x and y coordinates of center are mixed up here.

how can we get it?

I was wondering if there is a way to perform inference on a single random face image. Are there any pretrained weights available for this purpose? I noticed the weights link in the readme but I'm unsure how to use them for inference on a single image. Additionally, what preprocessing steps do I need to perform on an image before passing it to the model?

Hello, I tried to reproduce the training process. During the data preprocessing stage, I took one frame of a video every 10 frames and obtained 210000 photos. However, this result seems to be not as good. I spent a day training 15 epochs, but the accuracy remained between 0.75 and 0.85. Is it because I trained too little or something else? Does anyone have the same outcome as me?

I used a random video and created it's ldm.json and then run the test and finally received following response (I actually had to put another video in orignal folder to make it work):

Loaded pretrained weights for efficientnet-b3 Load deepfake dataset from ./test_images.. /usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py:558: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary. warnings.warn(_create_warning_msg( [ WARN:[email protected]] global loadsave.cpp:241 findDecoder imread_('./test_images/original_sequences/youtube/raw/frames/video/frame_0.png'): can't open/read file: check file path/integrity Frame-AUC of FF++ is 0.0000 Video-AUC of FF++ is 0.0000

I'm unable to understand what does Frame-AUC and Video-AUC represents, I would really appreciate any kind response.

I am a beginner, and I would like to know how to import the "jpegpy" module. Do I need to compile it? If so, how should I compile it?
thanks!

Hello,

Is there a script available for running inference on individual video files, separate from evaluating datasets using test.py?

Currently, test.py relies heavily on DeepfakeDataset and requires the use of a dataset for evaluation. I would like to request a dedicated script for running inference on individual video files or perhaps some guidance on how to proceed.

For instance, it would be helpful to have an inference script that can be executed using the following command:
python3 test.py test_video.mp4 --config caddm_test.cfg

and output to be:

Inference results for test_video.mp4:
Frame predictions: [0.823, 0.567, 0.912, 0.187, ...]
Average prediction: 0.654

Are there any plans for this?

测试集和训练集需要自己手动划分吗

I want to use this network to detect fake image data sets instead of video data sets. How do I modify the code? Also, with a new dataset, how do I generate a json file for the dataset?

I would appreciate your advice from the experts