Skip to content

Commit eba81e9

Browse files
kprokofikprokofi
committed
updated state from OpenVINO
1 parent a848aee commit eba81e9

33 files changed

+883
-866
lines changed

‎README.md

+7-5
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22
Towards the solving anti-spoofing problem on RGB only data.
33
## Introduction
44
This repository contains a training and evaluation pipeline with different regularization methods for face anti-spoofing network. There are a few models available for training purposes, based on MobileNetv2 (MN2) and MobileNetv3 (MN3). Project supports natively three datasets: [CelebA Spoof](https://github.com/Davidzhangyuanhan/CelebA-Spoof), [LCC FASD](https://csit.am/2019/proceedings/PRIP/PRIP3.pdf), [CASIA-SURF CeFA](https://arxiv.org/pdf/2003.05136.pdf). Also, you may want to train or validate with your own data. Final model based on MN3 trained on the CelebA Spoof dataset. The model has 3.72 times fewer parameters and 24.3 times fewer GFlops than AENET from the original paper, at the same time MN3 better generalizes on cross-domain. The code contains a demo that you can launch in real-time with your webcam or on the provided video. You can check out the short video on how it works on the [goole drive](https://drive.google.com/drive/u/0/folders/1A6wa3AlrdjyNPkXT81knIzXxR7SAYm1q). Also, the code supports conversion to the ONNX format.
5+
You can follow the links to the configuration files with smaller models to train them as-is and obtain metrics below.
6+
57
| model name | dataset | AUC | EER% | APCER% | BPCER% | ACER% | MParam | GFlops | Link to snapshot |
68
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
79
| MN3_large |CelebA-Spoof| 0.998 | 2.26 | 0.69 | 6.92 | 3.8 | 3.02 | 0.15 | [snapshot](https://drive.google.com/drive/u/0/folders/1A6wa3AlrdjyNPkXT81knIzXxR7SAYm1q) |
@@ -70,11 +72,11 @@ The script for training and inference uses a configuration file. This is [defaul
7072
* **resize** - resize of the image
7173
* **checkpoint** - the name of the checkpoint to save and the path to the experiment folder where checkpoint, tensorboard logs and eval metrics will be kept
7274
* **loss** - there are available two possible losses: `amsoftmax` with `cos`, `arcos`, `cross_enropy` margins and `soft_triple` with different number of inner classes. For more details about this soft triple loss see in [paper](https://arxiv.org/pdf/1909.05235.pdf)
73-
* **loss.amsoftmax.ratio**a - there is availability to use different m for different classes. The ratio is the weights on which provided `m` will be divided for a specific class. For example ratio = [1,2] means that m for the first class will equal to m, but for the second will equal to m/2
75+
* **loss.amsoftmax.ratio** - there is availability to use different m for different classes. The ratio is the weights on which provided `m` will be divided for a specific class. For example ratio = [1,2] means that m for the first class will equal to m, but for the second will equal to m/2
7476
* **loss.amsoftmax.gamma** - if this constant differs from 0 then the focal loss will be switched on with the corresponding gamma
75-
* **For soft triple loss**: `Cn` - number of classes, `K` - number of proxies for each class, `tau` - parameter for regularisation number of proxies
76-
* **model** - there are parameters concern model. `pretrained` means that you want to train with the imagenet weights (you can download weights from [google drive](https://drive.google.com/drive/u/0/folders/1A6wa3AlrdjyNPkXT81knIzXxR7SAYm1q) and specify the path to it in the `imagenet weights` parameter. **model_type** - type of the model, 'Mobilenet3' and 'Mobilenet2' are available. **size** param means the size of the mobilenetv3, there are 'large' and 'small' options. Note that this will change mobilenev3 only. **embeding_dim** - the size of the embeding (vector of features after average pooling). **width_mult** - the width scaling parameter of the model. Note, that you will need the appropriate imagenet weights if you want to train your model with transfer learning. On google drive weights with 0.75, 1.0 value of this parameter is available
77-
* **aug** - advanced augmentation, appropriate value for type is 'cutmix' or 'mixup. lambda = BetaDistribution(alpha, beta), cutmix_prob - probability of applying cutmix on image.
77+
* **For soft triple loss**: `Cn` - number of classes, `K` - number of proxies for each class, `tau` - parameter for regularization number of proxies
78+
* **model** - there are parameters concerning model. `pretrained` means that you want to train with the imagenet weights (you can download weights from [google drive](https://drive.google.com/drive/u/0/folders/1A6wa3AlrdjyNPkXT81knIzXxR7SAYm1q) and specify the path to it in the `imagenet weights` parameter. **model_type** - type of the model, 'Mobilenet3' and 'Mobilenet2' are available. **size** param means the size of the mobilenetv3, there are 'large' and 'small' options. Note that this will change mobilenev3 only. **embeding_dim** - the size of the embeding (vector of features after average pooling). **width_mult** - the width scaling parameter of the model. Note, that you will need the appropriate imagenet weights if you want to train your model with transfer learning. On google drive weights with 0.75, 1.0 values of this parameter are available
79+
* **aug** - there are some advanced augmentations are available. You can specify `cutmix` or `mixup` and appropriate params for them. `alpha` and `beta` are used for choosing `lambda` from beta distribution, `aug_prob` response for the probability of applying augmentation on the image.
7880
* **curves** - you can specify the name of the curves, then set option `--draw_graph` to `True` when evaluating with eval_protocol.py script
7981
* **dropout** - `bernoulli` and `gaussian` dropouts are available with respective parameters
8082
* **data_parallel** - you can train your network on several GPU
@@ -120,7 +122,7 @@ You will see the mean difference (L1 metric distance) on the first and second pr
120122

121123
## Demo
122124
![demo.png](./demo/demo.png)
123-
To start demo you need to [download] OpenVINO™ face detector model. Concretely, you will need `face-detection-0100` version.
125+
To start demo you need to [download] OpenVINO™ face detector model. Concretely, you will need `face-detection-0100` version.
124126
On [google drive](https://drive.google.com/drive/u/0/folders/1A6wa3AlrdjyNPkXT81knIzXxR7SAYm1q) you will see a trained antispoofing model that you can download and run, or choose your own trained model. Use OpenVINO™ format to obtain the best performance speed, but PyTorch format will work as well.
125127

126128
After preparation start demo by running:

‎compute_mean_std.py

+14-23
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,20 @@
1-
'''MIT License
2-
3-
Copyright (C) 2020 Prokofiev Kirill
4-
5-
Permission is hereby granted, free of charge, to any person obtaining a copy
6-
of this software and associated documentation files (the "Software"),
7-
to deal in the Software without restriction, including without limitation
8-
the rights to use, copy, modify, merge, publish, distribute, sublicense,
9-
and/or sell copies of the Software, and to permit persons to whom
10-
the Software is furnished to do so, subject to the following conditions:
11-
12-
The above copyright notice and this permission notice shall be included
13-
in all copies or substantial portions of the Software.
14-
15-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
16-
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17-
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
18-
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
19-
OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
20-
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE
21-
OR OTHER DEALINGS IN THE SOFTWARE.'''
1+
"""
2+
Copyright (c) 2020 Intel Corporation
3+
Licensed under the Apache License, Version 2.0 (the "License");
4+
you may not use this file except in compliance with the License.
5+
You may obtain a copy of the License at
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
Unless required by applicable law or agreed to in writing, software
8+
distributed under the License is distributed on an "AS IS" BASIS,
9+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
10+
See the License for the specific language governing permissions and
11+
limitations under the License.
12+
"""
2213

2314
import argparse
2415

2516
import albumentations as A
26-
import cv2
17+
import cv2 as cv
2718
import torch
2819
from torch.utils.data import DataLoader
2920
from tqdm import tqdm
@@ -41,7 +32,7 @@ def main():
4132
args = parser.parse_args()
4233
# transform image
4334
transforms = A.Compose([
44-
A.Resize(*args.img_size, interpolation=cv2.INTER_CUBIC),
35+
A.Resize(*args.img_size, interpolation=cv.INTER_CUBIC),
4536
A.Normalize(mean=[0, 0, 0], std=[1, 1, 1])
4637
])
4738
root_folder = args.root

‎configs/config.py

+3-4
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121

2222
scheduler = dict(milestones=[20,40], gamma=0.2)
2323

24-
data = dict( batch_size=256,
24+
data = dict(batch_size=256,
2525
data_loader_workers=4,
2626
sampler=None,
2727
pin_memory=True)
@@ -39,7 +39,6 @@
3939
smoothing=0.1,
4040
ratio=[1,1],
4141
gamma=0),
42-
4342
soft_triple=dict(cN=2, K=10, s=1, tau=.2, m=0.35))
4443

4544
epochs = dict(start_epoch=0, max_epoch=71)
@@ -54,13 +53,13 @@
5453
aug = dict(type_aug=None,
5554
alpha=0.5,
5655
beta=0.5,
57-
cutmix_prob=0.7)
56+
aug_prob=0.7)
5857

5958
curves = dict(det_curve='det_curve_0.png',
6059
roc_curve='roc_curve_0.png')
6160

6261
dropout = dict(prob_dropout=0.1,
63-
classifier=0.5,
62+
classifier=0.35,
6463
type='bernoulli',
6564
mu=0.5,
6665
sigma=0.3)

‎configs/config_large_075.py

+76
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
exp_num = 0
2+
3+
dataset = 'celeba_spoof'
4+
5+
multi_task_learning = True
6+
7+
evaluation = True
8+
9+
test_steps = None
10+
11+
datasets = dict(LCCFASD_root='./LCC_FASDcropped',
12+
Celeba_root='./CelebA_Spoof',
13+
Casia_root='./CASIA')
14+
15+
external = dict(train=dict(), val=dict(), test=dict())
16+
17+
img_norm_cfg = dict(mean=[0.5931, 0.4690, 0.4229],
18+
std=[0.2471, 0.2214, 0.2157])
19+
20+
optimizer = dict(lr=0.005, momentum=0.9, weight_decay=5e-4)
21+
22+
scheduler = dict(milestones=[20,50], gamma=0.2)
23+
24+
data = dict(batch_size=256,
25+
data_loader_workers=4,
26+
sampler=None,
27+
pin_memory=True)
28+
29+
resize = dict(height=128, width=128)
30+
31+
checkpoint = dict(snapshot_name="MobileNet3.pth.tar",
32+
experiment_path='./logs')
33+
34+
loss = dict(loss_type='amsoftmax',
35+
amsoftmax=dict(m=0.5,
36+
s=1,
37+
margin_type='cross_entropy',
38+
label_smooth=False,
39+
smoothing=0.1,
40+
ratio=[1,1],
41+
gamma=0),
42+
soft_triple=dict(cN=2, K=10, s=1, tau=.2, m=0.35))
43+
44+
epochs = dict(start_epoch=0, max_epoch=71)
45+
46+
model= dict(model_type='Mobilenet3',
47+
model_size = 'large',
48+
width_mult = 1.0,
49+
pretrained=True,
50+
embeding_dim=1280,
51+
imagenet_weights='./pretrained/mobilenetv3-large-0.75-9632d2a8.pth')
52+
53+
aug = dict(type_aug=None,
54+
alpha=0.5,
55+
beta=0.5,
56+
aug_prob=0.7)
57+
58+
curves = dict(det_curve='det_curve_0.png',
59+
roc_curve='roc_curve_0.png')
60+
61+
dropout = dict(prob_dropout=0.1,
62+
classifier=0.3,
63+
type='bernoulli',
64+
mu=0.5,
65+
sigma=0.3)
66+
67+
data_parallel = dict(use_parallel=False,
68+
parallel_params=dict(device_ids=[0,1], output_device=0))
69+
70+
RSC = dict(use_rsc=False,
71+
p=0.333,
72+
b=0.333)
73+
74+
test_dataset = dict(type='LCC_FASD')
75+
76+
conv_cd = dict(theta=0)

‎configs/config_small.py

+76
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
exp_num = 0
2+
3+
dataset = 'celeba_spoof'
4+
5+
multi_task_learning = True
6+
7+
evaluation = True
8+
9+
test_steps = None
10+
11+
datasets = dict(LCCFASD_root='./LCC_FASDcropped',
12+
Celeba_root='./CelebA_Spoof',
13+
Casia_root='./CASIA')
14+
15+
external = dict(train=dict(), val=dict(), test=dict())
16+
17+
img_norm_cfg = dict(mean=[0.5931, 0.4690, 0.4229],
18+
std=[0.2471, 0.2214, 0.2157])
19+
20+
optimizer = dict(lr=0.005, momentum=0.9, weight_decay=5e-4)
21+
22+
scheduler = dict(milestones=[20,50], gamma=0.2)
23+
24+
data = dict(batch_size=256,
25+
data_loader_workers=4,
26+
sampler=None,
27+
pin_memory=True)
28+
29+
resize = dict(height=128, width=128)
30+
31+
checkpoint = dict(snapshot_name="MobileNet3.pth.tar",
32+
experiment_path='./logs')
33+
34+
loss = dict(loss_type='amsoftmax',
35+
amsoftmax=dict(m=0.5,
36+
s=1,
37+
margin_type='cross_entropy',
38+
label_smooth=False,
39+
smoothing=0.1,
40+
ratio=[1,1],
41+
gamma=0),
42+
soft_triple=dict(cN=2, K=10, s=1, tau=.2, m=0.35))
43+
44+
epochs = dict(start_epoch=0, max_epoch=71)
45+
46+
model= dict(model_type='Mobilenet3',
47+
model_size = 'large',
48+
width_mult = 1.0,
49+
pretrained=True,
50+
embeding_dim=1024,
51+
imagenet_weights='./pretrained/mobilenetv3-small-55df8e1f.pth')
52+
53+
aug = dict(type_aug=None,
54+
alpha=0.5,
55+
beta=0.5,
56+
aug_prob=0.7)
57+
58+
curves = dict(det_curve='det_curve_0.png',
59+
roc_curve='roc_curve_0.png')
60+
61+
dropout = dict(prob_dropout=0.1,
62+
classifier=0.1,
63+
type='bernoulli',
64+
mu=0.5,
65+
sigma=0.3)
66+
67+
data_parallel = dict(use_parallel=False,
68+
parallel_params=dict(device_ids=[0,1], output_device=0))
69+
70+
RSC = dict(use_rsc=False,
71+
p=0.333,
72+
b=0.333)
73+
74+
test_dataset = dict(type='LCC_FASD')
75+
76+
conv_cd = dict(theta=0)

‎configs/config_small_075.py

+76
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
exp_num = 0
2+
3+
dataset = 'celeba_spoof'
4+
5+
multi_task_learning = True
6+
7+
evaluation = True
8+
9+
test_steps = None
10+
11+
datasets = dict(LCCFASD_root='./LCC_FASDcropped',
12+
Celeba_root='./CelebA_Spoof',
13+
Casia_root='./CASIA')
14+
15+
external = dict(train=dict(), val=dict(), test=dict())
16+
17+
img_norm_cfg = dict(mean=[0.5931, 0.4690, 0.4229],
18+
std=[0.2471, 0.2214, 0.2157])
19+
20+
optimizer = dict(lr=0.005, momentum=0.9, weight_decay=5e-4)
21+
22+
scheduler = dict(milestones=[20,50], gamma=0.2)
23+
24+
data = dict(batch_size=256,
25+
data_loader_workers=4,
26+
sampler=None,
27+
pin_memory=True)
28+
29+
resize = dict(height=128, width=128)
30+
31+
checkpoint = dict(snapshot_name="MobileNet3.pth.tar",
32+
experiment_path='./logs')
33+
34+
loss = dict(loss_type='amsoftmax',
35+
amsoftmax=dict(m=0.5,
36+
s=1,
37+
margin_type='cross_entropy',
38+
label_smooth=False,
39+
smoothing=0.1,
40+
ratio=[1,1],
41+
gamma=0),
42+
soft_triple=dict(cN=2, K=10, s=1, tau=.2, m=0.35))
43+
44+
epochs = dict(start_epoch=0, max_epoch=71)
45+
46+
model= dict(model_type='Mobilenet3',
47+
model_size = 'large',
48+
width_mult = 0.75,
49+
pretrained=True,
50+
embeding_dim=1024,
51+
imagenet_weights='./pretrained/mobilenetv3-small-0.75-86c972c3.pth')
52+
53+
aug = dict(type_aug=None,
54+
alpha=0.5,
55+
beta=0.5,
56+
aug_prob=0.7)
57+
58+
curves = dict(det_curve='det_curve_0.png',
59+
roc_curve='roc_curve_0.png')
60+
61+
dropout = dict(prob_dropout=0.1,
62+
classifier=0.35,
63+
type='bernoulli',
64+
mu=0.5,
65+
sigma=0.3)
66+
67+
data_parallel = dict(use_parallel=False,
68+
parallel_params=dict(device_ids=[0,1], output_device=0))
69+
70+
RSC = dict(use_rsc=False,
71+
p=0.333,
72+
b=0.333)
73+
74+
test_dataset = dict(type='LCC_FASD')
75+
76+
conv_cd = dict(theta=0)

‎conversion_checker.py

+13-22
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,15 @@
1-
'''MIT License
2-
3-
Copyright (C) 2020 Prokofiev Kirill
4-
5-
Permission is hereby granted, free of charge, to any person obtaining a copy
6-
of this software and associated documentation files (the "Software"),
7-
to deal in the Software without restriction, including without limitation
8-
the rights to use, copy, modify, merge, publish, distribute, sublicense,
9-
and/or sell copies of the Software, and to permit persons to whom
10-
the Software is furnished to do so, subject to the following conditions:
11-
12-
The above copyright notice and this permission notice shall be included
13-
in all copies or substantial portions of the Software.
14-
15-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
16-
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17-
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
18-
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
19-
OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
20-
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE
21-
OR OTHER DEALINGS IN THE SOFTWARE.'''
1+
"""
2+
Copyright (c) 2020 Intel Corporation
3+
Licensed under the Apache License, Version 2.0 (the "License");
4+
you may not use this file except in compliance with the License.
5+
You may obtain a copy of the License at
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
Unless required by applicable law or agreed to in writing, software
8+
distributed under the License is distributed on an "AS IS" BASIS,
9+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
10+
See the License for the specific language governing permissions and
11+
limitations under the License.
12+
"""
2213

2314
import argparse
2415
import inspect
@@ -67,7 +58,7 @@ def pred_spoof(batch, spoof_model_torch, spoof_model_openvino):
6758
return output1, output2
6859

6960
def check_accuracy(torch_pred, openvino_pred):
70-
diff = abs(np.array(openvino_pred) - np.array(torch_pred))
61+
diff = np.abs(openvino_pred - torch_pred)
7162
avg = diff.mean(axis=0)
7263
return avg
7364

0 commit comments

Comments
 (0)