Hi, thanks for the code!

I encountered an issue when reading the TRU implementation. I notice that during the calculation of Covariance Matrix, mean value "mu" were not subtracted from x_sub and x_sub_t before performing multiplication. It seems to be different from the original paper where "mu" needs to be subtracted to get the correct Covariance Matrix.

Could you help to shed some lights on this issue?

Hello and thank you for your amazing work.
Could you please provide a simple test script to classify a single image?

Hi
how to run this script??
when I working the train.py
it some make error...

$ python3 train.py
2020-07-13 14:39:57.524805: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-07-13 14:39:57.577527: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties:
pciBusID: 0000:8a:00.0 name: Tesla V100-SXM2-16GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 15.78GiB deviceMemoryBandwidth: 836.37GiB/s
2020-07-13 14:39:57.577764: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-07-13 14:39:57.579376: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-07-13 14:39:57.580883: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-07-13 14:39:57.581165: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-07-13 14:39:57.583019: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-07-13 14:39:57.583986: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-07-13 14:39:57.587860: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-07-13 14:39:57.590829: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
Traceback (most recent call last):
File "train.py", line 54, in
main()
File "train.py", line 38, in main
config = Config()
File "/home/intunit/LGJ/ZSFA/model/config.py", line 59, in init
tf.config.experimental.set_memory_growth(gpus[self.GPU_USAGE], True)
IndexError: list index out of range

Hi there,

I just downloaded the dataset from the link provided by SiW Management Team. Some of the binary files seems to have zero size which causes error. Could you please provide checksum for Binary.zip files.

Best,
HopeAI

我发现通过只改变 config.MODE 会出现问题
同时在训练过程画图的过程中 plotResults会出现
cv2.error: OpenCV(4.1.2) /io/opencv/modules/imgproc/src/color.simd_helpers.hpp:92: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<3, 4>; VDcn = cv::impl::{anonymous}::Set<3, 4>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = (cv::impl::::SizePolicy)2u; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'

Invalid number of channels in input image:
'VScn::contains(scn)'
where
'scn' is 2

Hello,
Thanks for sharing the code here. I wanted to know the format of the data to train. Tia.

Is label a normalized value between [-1,1]

or is it a unique idx per class
like
0 - Live
1 - Print
2 - Video

etc ?

dose the dataset have risk of infringement?

I'm getting the following error when loading the Siw-M dataset, can anyone help me?

I had to modify dataset.py so that the files were loaded correctly:

for _dir in data_dir: for subdir in os.listdir(_dir): _list = glob.glob(_dir+subdir+'/*.dat') data_samples += _list

This is the error I get when I try to train:

ValueError: Dimensions must be equal, but are 1568 and 2048 for '{{node tru/linear/cond/sub}} = Sub[T=DT_FLOAT](tru/linear/cond/GatherNd, tru/linear/cond/StopGradient)' with input shapes: [?,1568], [1,2048].

I believe you have already known this problem exists. I just wanna ask if you can suggest any solutions

 dmap1 = dmap * (1-label);
 dmap2 = np.ones_like(dmap) * label;
 dmap = np.concatenate([dmap1, dmap2], axis=2); 

Why do we need two depth maps。
@yaojieliu

Hi,

I'm trying to run your example with my custom dataset, as I can see in dataset.py, you read image's RGB and depth map from .dat file.
So does this approach needs 2 modal types of image: RGB and Depth for training?

Thanks

1- How long did the training take?
2- Which GPU card did you use?
3- Have you trained as follows?

config.STEPS_PER_EPOCH = 1000
config.MAX_EPOCH = 40
config.BATCH_SIZE = 32

4- What are the final loss function values after training ends?

Map loss = 0.001, ???
Cls loss = 0.001, ???

Thanks in advanced

Let's say I have only facial images. How do I evaluate this as fake and real when I put this into the model? Could you explain this through the code? Can I use "leaf_l1_score" ? Threshold?

def train_one_step(self, data_batch, step, training):
.....
with tf.GradientTape() as tape:
dmap_pred, cls_pred, route_value, leaf_node_mask, tru_loss, mu_update, eigenvalue, trace =
dtn(image, labels, False) (default :True)
.......

dmap_pred = ?
cls_pred =?
route_value =?
leaf_node_mask =?

def leaf_l1_score(xlist, masklist, ch=None):
loss_list = []
xshape = xlist[0].shape
scores = []
for x, mask in zip(xlist, masklist):
if ch is not None:
score = tf.reduce_mean(tf.reshape(tf.abs(x[:, :, :, ch]), [xshape[0], -1]), axis=1)
else:
score = tf.reduce_mean(tf.reshape(tf.abs(x), [xshape[0], -1]), axis=1)
spoof_score = score * mask[:, 0]
scores.append(spoof_score)
loss = np.sum(np.stack(scores, axis=1), axis=1)
return loss

Thanks in advanced

Hello,
Thanks for yours code.I has a question that in Tree Routing Unit (TRU), the vector V is invariant when train？

Great tutorial! how can I deploy it to mobile phone?
Thanks.

Traceback (most recent call last):
File "/home/littlea/CVPR2019-DeepTreeLearningForZeroShotFaceAntispoofing/train.py", line 48, in
main()
File "/home/littlea/CVPR2019-DeepTreeLearningForZeroShotFaceAntispoofing/train.py", line 46, in main
model.train(dataset_train, None)
File "/home/littlea/CVPR2019-DeepTreeLearningForZeroShotFaceAntispoofing/model/model.py", line 229, in train
self.train_one_step(next(it), global_step, True)
File "/home/littlea/CVPR2019-DeepTreeLearningForZeroShotFaceAntispoofing/model/model.py", line 288, in train_one_step
depth_map_loss = leaf_l1_loss(dmap_pred, tf.image.resize(dmap, [32, 32]), leaf_node_mask)
File "/home/littlea/CVPR2019-DeepTreeLearningForZeroShotFaceAntispoofing/model/loss.py", line 50, in leaf_l1_loss
loss = tf.reduce_mean(tf.reshape(tf.abs(x-y), [xshape[0], -1]), axis=1)
File "/home/littlea/anaconda3/envs/tensorflow/lib/python3.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 7715, in reshape
"Reshape", tensor=tensor, shape=shape, name=name)
File "/home/littlea/anaconda3/envs/tensorflow/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py", line 530, in _apply_op_helper
raise err
File "/home/littlea/anaconda3/envs/tensorflow/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py", line 527, in _apply_op_helper
preferred_dtype=default_dtype)
File "/home/littlea/anaconda3/envs/tensorflow/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1224, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/home/littlea/anaconda3/envs/tensorflow/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py", line 305, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/home/littlea/anaconda3/envs/tensorflow/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py", line 246, in constant
allow_broadcast=True)
File "/home/littlea/anaconda3/envs/tensorflow/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py", line 284, in _constant_impl
allow_broadcast=allow_broadcast))
File "/home/littlea/anaconda3/envs/tensorflow/lib/python3.7/site-packages/tensorflow/python/framework/tensor_util.py", line 562, in make_tensor_proto
"supported type." % (type(values), values))
TypeError: Failed to convert object of type <class 'list'> to Tensor. Contents: [Dimension(None), -1]. Consider casting elements to a supported type.

Is there any possibility to receive pre-trained models for this project?
The project looks really interesting and can be a good way to take research forward in the area of face spoof detection.

Thank you!

The GitHub code just doesn't follow the approach/architecture described in the paper.
E.g. -

1. Github: Image input is RGB
   Paper: Image input is RGB+HSV
2. SFL architecture is completely different

I have a long list of mismatches, please explain why is this so and if you are planning to release code utilized in paper.

I have the SiW-M dataset with me but it has a dat file corresponding to each video. That means it has (No. of frames) * (256*256*3+64*64+1) bytes. So did you extract all frames from each dat file and trained on all of them?

Hi,
I tried to access http://cvlab.cse.msu.edu/siw-m-spoof-in-the-wild-with-multiple-attacks-database.html to download SiW-M dataset, but got 404 Not Found. Can you help me find out the reason?

Hey. Is there somewhere that I can find how to structure the data within the project itself? I saw it doesnt come with a dataset attached and I was just wondering how I would go about creating a dataset that would work with training the model and where/how would I implement it?

I read through the paper and couldnt find any specifics in there either.