Scalable Vision Transformers with Hierarchical Pooling arxiv
PaddlePaddle training/validation code and pretrained models for HVT.
The official pytorch implementation is here.
This implementation is developed by PaddleViT.
- Update (2021-12-28): Code is released and ported weights are uploaded.
| Model | Acc@1 | Acc@5 | #Params | FLOPs | Image Size | Crop_pct | Interpolation | Link |
|---|---|---|---|---|---|---|---|---|
| HVT-Ti-1 | 69.45 | 89.28 | 5.7M | 0.6G | 224 | 0.875 | bicubic | google/baidu(egds) |
| HVT-S-0 | 80.30 | 95.15 | 22.0M | 4.6G | 224 | 0.875 | bicubic | google/baidu(hj7a) |
| HVT-S-1 | 78.06 | 93.84 | 22.1M | 2.4G | 224 | 0.875 | bicubic | google/baidu(tva8) |
| HVT-S-2 | 77.41 | 93.48 | 22.1M | 1.9G | 224 | 0.875 | bicubic | google/baidu(bajp) |
| HVT-S-3 | 76.30 | 92.88 | 22.1M | 1.6G | 224 | 0.875 | bicubic | google/baidu(rjch) |
| HVT-S-4 | 75.21 | 92.34 | 22.1M | 1.6G | 224 | 0.875 | bicubic | google/baidu(ki4j) |
*The results are evaluated on ImageNet2012 validation set.
We provide a few notebooks in aistudio to help you get started:
*(coming soon)*
- Python>=3.6
- yaml>=0.2.5
- PaddlePaddle>=2.1.0
- yacs>=0.1.8
ImageNet2012 dataset is used in the following folder structure:
│imagenet/
├──train/
│ ├── n01440764
│ │ ├── n01440764_10026.JPEG
│ │ ├── n01440764_10027.JPEG
│ │ ├── ......
│ ├── ......
├──val/
│ ├── n01440764
│ │ ├── ILSVRC2012_val_00000293.JPEG
│ │ ├── ILSVRC2012_val_00002138.JPEG
│ │ ├── ......
│ ├── ......
To use the model with pretrained weights, download the .pdparam weight file and change related file paths in the following python scripts. The model config files are located in ./configs/.
For example, assume the downloaded weight file is stored in ./hvt_s2_patch16_224.pdparams, to use the hvt_s2_patch16_224 model in python:
from config import get_config
from hvt import build_hvt as build_model
# config files in ./configs/
config = get_config('./configs/hvt_s2_patch16_224.yaml')
# build model
model = build_model(config)
# load pretrained weights, .pdparams is NOT needed
model_state_dict = paddle.load('./hvt_s2_patch16_224')
model.set_dict(model_state_dict)To evaluate HVT model performance on ImageNet2012 with a single GPU, run the following script using command line:
sh run_eval.shor
CUDA_VISIBLE_DEVICES=0 \
python main_single_gpu.py \
-cfg='./configs/hvt_s2_patch16_224.yaml' \
-dataset='imagenet2012' \
-batch_size=16 \
-data_path='/dataset/imagenet' \
-eval \
-pretrained='./hvt_s2_patch16_224'Run evaluation using multi-GPUs:
sh run_eval_multi.shor
CUDA_VISIBLE_DEVICES=0,1,2,3 \
python main_multi_gpu.py \
-cfg='./configs/hvt_s2_patch16_224.yaml' \
-dataset='imagenet2012' \
-batch_size=16 \
-data_path='/dataset/imagenet' \
-eval \
-pretrained='./hvt_s2_patch16_224'To train the HVT Transformer model on ImageNet2012 with single GPU, run the following script using command line:
sh run_train.shor
CUDA_VISIBLE_DEVICES=0 \
python main_single_gpu.py \
-cfg='./configs/hvt_s2_patch16_224.yaml' \
-dataset='imagenet2012' \
-batch_size=16 \
-data_path='/dataset/imagenet' Run training using multi-GPUs:
sh run_train_multi.shor
CUDA_VISIBLE_DEVICES=0,1,2,3 \
python main_multi_gpu.py \
-cfg='./configs/hvt_s2_patch16_224.yaml' \
-dataset='imagenet2012' \
-batch_size=16 \
-data_path='/dataset/imagenet'
@inproceedings{pan2021scalable,
title={Scalable vision transformers with hierarchical pooling},
author={Pan, Zizheng and Zhuang, Bohan and Liu, Jing and He, Haoyu and Cai, Jianfei},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
pages={377--386},
year={2021}
}