seutao / facebagnet Goto Github PK

FaceBagNet - Patch-based Methods for Multi-modal Face Anti-spoofing (FAS)

Python 100.00%

facebagnet's Introduction

Patch-based Methods for Face Anti-spoofing (FAS) & Code for CVPR2019 FAS Attack Detection Challenge

This is the source code for 2nd palce solution to the ChaLearn Face Anti-spoofing Attack Detection Challenge hosted by ChaLearn.

Recent Update

2021.11.24: Add ViT for patch-based FAS

2021.10.12: Add VisionPermutator, MLPMixer and ConvMixer for patch-based FAS

2019.3.10: Code upload for the origanizers to reproduce.

Dependencies

imgaug==0.4.0
torch==1.9.0
torchvision==0.10.0

Pretrained models

download [models.2021]

CASIA-SURF validation score (ACER)

Single-modal Model	Color	Depth	ir
FaceBagNet	0.0672	0.0036	0.1003
ConvMixer	0.0311	0.0025	0.1073
MLPMixer	0.0584	0.0010	0.2382
VisionPermutator(ViP)	0.0570	0.0304	0.2571
VisonTransformer(ViT)	0.0683	0.0036	0.2799

Multi-modal Model	patch size 32	patch size 48	patch size 64
FaceBagNetFusion	0.0009	0.0006	0.0007
ViTFusion	0.0169	0.0778	0.0375

Train single-modal Model

train FaceBagNet with color imgs， patch size 48：

CUDA_VISIBLE_DEVICES=0 python train.py --model=FaceBagNet --image_mode=color --image_size=48

infer

CUDA_VISIBLE_DEVICES=0 python train.py --mode=infer_test --model=FaceBagNet --image_mode=color --image_size=48

Train multi-modal fusion model

train FaceBagNet fusion model with multi-modal imgs， patch size 48：

CUDA_VISIBLE_DEVICES=0 python train_fusion.py --model=FaceBagNetFusion --image_size=48

infer

CUDA_VISIBLE_DEVICES=0 python train_fusion.py --mode=infer_test --model=FaceBagNet --image_size=48

ViT for Multi-modal Face Anti-spoofing

CUDA_VISIBLE_DEVICES=0 python train_fusion.py --model=ViTFusion --image_size=96 --image_patch 16

Citation

If you find this work or code is helpful in your research, please cite:

@InProceedings{Shen_2019_CVPR_Workshops,
author = {Shen, Tao and Huang, Yuyu and Tong, Zhijun},
title = {FaceBagNet: Bag-Of-Local-Features Model for Multi-Modal Face Anti-Spoofing},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2019}
}

Contact

If you have any questions, feel free to E-mail me via: [email protected]

facebagnet's People

Contributors

Stargazers

Watchers

Forkers

mike07026 gds101054108 qianji13 lonelygo pineking shubhampachori12110095 qaz734913414 sermakarevich makhthum abhinavm24 zllrunning jentman leo-xxx tiansong1991 kingfaluis oujieww dejavusans piseyyou hxhh gzzgz 5059 jacke121 miwaliu mikekiwa cosmoshua cskericchan jstzwjr yuemengyuan tanghy2016 wuxiaolianggit zhangtaozhir huangyuyu0426 bholdmanny chetanseth aliushn chanbluky qing0991 jarvizz mfzhang elijah-yi baucheng kk52099 peili-liang xiaoye77 sndler zhengshunjie shory152 xiaomitop skjack 7aughing slzhly wuqiangch dfenglei ewail zhangjiekui airob yudie433 jrznbu luckydoggy liyuejul gpfworld csgcmai fzwei-yyds abrams90 boragocode quxiaofeng hdjsjyl yangsenwxy vareto-forks zhangjf2018 wmorders hx121071 zhukkang guoxpl xxyqsy yjingyu tloki hell-to-heaven lj0620 allensmile seanyu-git jiayoulixiaoshuang paul0m chongchuanbing sprinterzzj aixinneucore class8hawk baymin0220 jizhi mogebuyi inghyun verigle raftliu zhleternity vzhangmeng726 shiyuan0806 jackweiwang david9998 lihuikenny pro-flynn

facebagnet's Issues

error: (-215:Assertion failed) !ssize.empty() in function 'resize'

作者你好，感谢你的开源，但我在运行代码的时候遇到了如下问题，请问你可以给我一些指导吗？谢谢～
Traceback (most recent call last):
in getitem
image = cv2.resize(image,(RESIZE_SIZE,RESIZE_SIZE))
cv2.error: OpenCV(3.4.2) /tmp/build/80754af9/opencv-suite_1535558553474/work/modules/imgproc/src/resize.cpp:4044: error: (-215:Assertion failed) !ssize.empty() in function 'resize'

test code？？

ModuleNotFoundError: No module named 'model.backbone.senet'

你好，这部分代码没有吗？

when will you provide the pre-trained model?

when will you provide the pre-trained model corresponding to this paper?

怎么判断图片是否有摩尔纹

模型是利用有摩尔纹跟没有摩尔纹数据集进行训练,这是人为区分出来的.
如果不人为区分,有什么方法可以判断一张图片是否有摩尔纹,是否有什么指标判断摩尔纹程度?

单IR测试结果和原论文不符

你好，我看原论文中单IR 48*48的结果TPR@FPR 10E-4=98.6，但是我加载您的预训练模型测试的结果为ACER : 0.0110,TPR@FPR=10E-2 :0.9872,TPR@FPR=10E-2 :0.9872,TPR@FPR=10E-3 :0.8242,TPR@FPR=10E-4 :0.5246

@SeuTao @huangyuyu0426

no module named 'augmentation'

train using CUDA_VISIBLE_DEVICES=0 python train_Fusion_CyclicLR.py --model=model_A --image_size=48

如何获取训练数据

加入ViT后有相应的论文吗

您好，如题，如果有新发表文章的话，能否更新一下呢？谢谢。

Different mean of models (baseline, A, B, C)?

Can you tell me the meaning of models that you experiment in this repo?
I can't find it on your paper.

如何进行推理，检测一张图片是否为活体

如题！！！

代码中缺少文件 data_helper

执行train命令的时候，会出错：
ModuleNotFoundError: No module named 'data_helper'

No module named 'imgaug'

Transfer learning: shape mismatch in two keys when load_pretrain()

Hello hello!

Congrats on the outstanding project!!

I am trying to load the pretrained models provided but I run into the same shape mismatch error, always in keys encoder.last_linear.weight and encoder.last_linear.bias, no matter which .pth.tar I try to load (IR, depth or RGB; 32, 48 or 64).

The code:

bobo_A = Net()
path = "\model_A_color_64\checkpoint\global_min_acer_model.pth.tar"
bobo_A.load_pretrain(path)

Raises:

RuntimeError: Error(s) in loading state_dict for Net:
	size mismatch for encoder.last_linear.weight: copying a param with shape torch.Size([1000, 2048]) from checkpoint, the shape in current model is torch.Size([2, 2048]).
	size mismatch for encoder.last_linear.bias: copying a param with shape torch.Size([1000]) from checkpoint, the shape in current model is torch.Size([2]).

I am not being able to find a workaround. Any help?

Thanks!

IR模态在测试集上提升到了97.9%

加入了正常IR双目摄像头采集的数据集,修改了数据增强方法,IR模态在测试集(CASIA_SURF+mydata)上从85.8%提升到了97.9%.

Running models on edge applications

Hi, sorry to bother you but I have a question about coverting the model(.pth) to .pt. I want to run my models on Android but none of them works, so I was wondering do you know how to run my trained models on edge applications?

请问哪里能看到您的文章

如题，如果有发表文章的话，能否更新一下？谢谢

请问有大佬能跑通吗？需要改代码吗？

改完一个error又出现一个error，这份代码能正常跑吗？

多模态问题

请问您这个多模态都有哪几个模态呀？

About problem of random cropping

In line 86-88 of the process/data_fusion.py, it seems will select the patch of different modalities from different face regions.
I would like to ask is it designed on purpose? Why design it? I didn't see the relevant explanation in the paper.

模式擦除的代码实现和论文有出入

您好，您在论文中提到的模式擦除是随机选取一个模式的特征置0，但是代码中的实现时让其变为随机噪声，这里是我理解的问题还是代码实现和论文写确实有出入呢？

FaceBagNet中的type=A/B/C 三种类型有证明哪种最优么

在定义FaceBagNet时,传参type=A/B/C,三种类型有证明哪种最优么?我看提供的预训练模型只是A的.
另外实际测试中,IR测试,如果图比较亮,则很容易判断为真,比如我对着白纸拍摄,根据rgb扣除人脸,但是测试结果却为真.

单模式红外人脸测试不准？

您好，非常感谢你的项目给我很大的帮助，但是我用红外摄像机，红外图片检测出人脸后送入网络测试，不管是活体图片还是照片测试结果接近1，不知道是我的测试代码有问题，还是我的测试方法有问题。您能给出一些建议吗

about model

The training input of the model is the whole face,
and the local patches are not reflected

FileNotFoundError: [Errno 2] No such file or directory: '/data1/shentao/DATA/CVPR19_FaceAntiSpoofing/train_list.txt'

(CVPR19) (base) marco@pc:~/antiFaceSpoofing/CVPR19-Face-Anti-spoofing-master$  
  CUDA_VISIBLE_DEVICES=0 python3 train_Fusion_CyclicLR.py --model=model_A 
  --image_size=48
Namespace(batch_size=128, cycle_inter=50, cycle_num=10, image_mode='fusion',  
image_size=48, mode='train', model='model_A', pretrained_model=None, train_fold_index=-1)
    out_dir      = ./models/model_A_fusion_48

    <additional comments>
      ... xxx baseline  ... 

** dataset setting **
fold: -1
fusion
train
fold index set:  -1
Traceback (most recent call last):
  File "train_Fusion_CyclicLR.py", line 254, in <module>
    main(config)
  File "train_Fusion_CyclicLR.py", line 231, in main
    run_train(config)
  File "train_Fusion_CyclicLR.py", line 48, in run_train
    fold_index=config.train_fold_index)
  File "/home/marco/antiFaceSpoofing/CVPR19-Face-Anti-spoofing-master/process
/data_fusion.py", line 22, in __init__
    self.set_mode(self.mode,self.fold_index)
  File "/home/marco/antiFaceSpoofing/CVPR19-Face-Anti-spoofing-master/process
/data_fusion.py", line 41, in set_mode
    self.train_list = load_train_list()
  File "/home/marco/antiFaceSpoofing/CVPR19-Face-Anti-spoofing-master/process
/data_helper.py", line 13, in load_train_list
    f = open(DATA_ROOT + '/train_list.txt')
FileNotFoundError: [Errno 2] No such file or directory: '/data1/shentao
  /DATA/CVPR19_FaceAntiSpoofing/train_list.txt'

Because the challenge is still on going and there are no released results.
I want to find good solution for anti-spoofing so i want to know you got 2nd place or something in the challenge.

Thanks for your contribution