Destruction and Construction Learning for Fine-grained Image Recognition

By Yue Chen, Yalong Bai, Wei Zhang, Tao Mei

Special thanks to Yuanzhi Liang for code refactoring.

UPDATE Jun. 10, 2020

! Research Intern Position Opening. Please send your cv to baiyalong[AT]jd.com if you are interested.

• First Place in CVPR 2020 AliProducts Challenge: Large-scale Product Recognition

UPDATE Jun. 21, 2019

Our solution for the FGVC Challenge 2019 (The Sixth Workshop on Fine-Grained Visual Categorization in CVPR 2019) is updated!

With ensemble of several DCL based classification models, we won:

• First Place in iMaterialist Challenge on Product Recognition
• First Place in Fieldguide Challenge: Moths & Butterflies
• Second Place in iFood - 2019 at FGVC6

Introduction

This project is a DCL pytorch implementation of Destruction and Construction Learning for Fine-grained Image Recognition, CVPR2019.

Requirements

1. Python 3.6

2. Pytorch 0.4.0 or 0.4.1

3. CUDA 8.0 or higher

For docker environment:

docker pull pytorch/pytorch:0.4-cuda9-cudnn7-devel

For conda environment:

conda create --name DCL file conda_list.txt

For more backbone supports in DCL, please check pretrainmodels and install:

pip install pretrainedmodels

Datasets Prepare

1. Download correspond dataset to folder 'datasets'

2. Data organization: eg. CUB

   All the image data are in './datasets/CUB/data/' e.g. './datasets/CUB/data/*.jpg'

   The annotation files are in './datasets/CUB/anno/' e.g. './dataset/CUB/data/train.txt'

   In annotations:

   name_of_image.jpg label_num\n

   e.g. for CUB in repository:

   Black_Footed_Albatross_0009_34.jpg 0
   Black_Footed_Albatross_0014_89.jpg 0
   Laysan_Albatross_0044_784.jpg 1
   Sooty_Albatross_0021_796339.jpg 2
   ...

Some examples of datasets like CUB, Stanford Car, etc. are already given in our repository. You can use DCL to your datasets by simply converting annotations to train.txt/val.txt/test.txt and modify the class number in config.py as in line67: numcls=200.

Training

Run train.py to train DCL.

For training CUB / STCAR / AIR from scratch

python train.py --data CUB --epoch 360 --backbone resnet50 \
                    --tb 16 --tnw 16 --vb 512 --vnw 16 \
                    --lr 0.0008 --lr_step 60 \
                    --cls_lr_ratio 10 --start_epoch 0 \
                    --detail training_descibe --size 512 \
                    --crop 448 --cls_mul --swap_num 7 7

For training CUB / STCAR / AIR from trained checkpoint

python train.py --data CUB --epoch 360 --backbone resnet50 \
                    --tb 16 --tnw 16 --vb 512 --vnw 16 \
                    --lr 0.0008 --lr_step 60 \
                    --cls_lr_ratio 10 --start_epoch $LAST_EPOCH \
                    --detail training_descibe4checkpoint --size 512 \
                    --crop 448 --cls_mul --swap_num 7 7

For training FGVC product datasets from scratch

 python train.py --data product --epoch 60 --backbone senet154 \
                    --tb 96 --tnw 32 --vb 512 --vnw 32 \
                    --lr 0.01 --lr_step 12 \
                    --cls_lr_ratio 10 --start_epoch 0 \
                    --detail training_descibe --size 512 \
                    --crop 448 --cls_2 --swap_num 7 7

For training FGVC datasets from trained checkpoint

 python train.py --data product --epoch 60 --backbone senet154 \
                    --tb 96 --tnw 32 --vb 512 --vnw 32 \
                    --lr 0.01 --lr_step 12 \
                    --cls_lr_ratio 10 --start_epoch $LAST_EPOCH \
                    --detail training_descibe4checkpoint --size 512 \
                    --crop 448 --cls_2 --swap_num 7 7

To achieve the similar results of paper, please use the default parameter settings.

Citation

Please cite our CVPR19 paper if you use this codebase in your work:

@InProceedings{Chen_2019_CVPR,
author = {Chen, Yue and Bai, Yalong and Zhang, Wei and Mei, Tao},
title = {Destruction and Construction Learning for Fine-Grained Image Recognition},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}

Find our more recent work:

Look-into-Object: Self-supervised Structure Modeling for Object Recognition. CVPR2020 [pdf, Source Code]