Abstract: Few-Shot Learning (FSL) requires vision models to quickly adapt to brand-new classification tasks with a shift in task distribution. Understanding the difficulties posed by this task distribution shift is central to FSL. In this paper, we show that a simple channel-wise feature transformation may be the key to unraveling this secret from a channel perspective. When facing novel few-shot tasks in the test-time datasets, this transformation can greatly improve the generalization ability of learned image representations, while being agnostic to the choice of training algorithms and datasets. Through an in-depth analysis of this transformation, we find that the difficulty of representation transfer in FSL stems from the severe channel bias problem of image representations: channels may have different importance in different tasks, while convolutional neural networks are likely to be insensitive, or respond incorrectly to such a shift. This points out a core problem of the generalization ability of modern vision systems and needs further attention in the future. Our code is available at this https URL.

[ICML 2022] Channel Importance Matters in Few-Shot Image Classification

Paper link: https://arxiv.org/abs/2206.08126

Abstract

Few-Shot Learning (FSL) requires vision models to quickly adapt to brand-new classification tasks with a shift in task distribution. Understanding the difficulties posed by this task distribution shift is central to FSL. In this paper, we show that a simple channel-wise feature transformation may be the key to unraveling this secret from a channel perspective. When facing novel few-shot tasks in the test-time datasets, this transformation can greatly improve the generalization ability of learned image representations, while being agnostic to the choice of training algorithms and datasets. Through an in-depth analysis of this transformation, we find that the difficulty of representation transfer in FSL stems from the severe channel bias problem of image representations: channels may have different importance in different tasks, while convolutional neural networks are likely to be insensitive, or respond incorrectly to such a shift. This points out a core problem of the generalization ability of modern vision systems and needs further attention in the future.

Where to find the simple channel-wise transformtion

The single-line simple_transform function has been put inside each test-time classifier, e.g., see in architectures/classifier/proto_head.py.

Installation

A list of packages to install is available in requirements.txt, just install them using command:

pip install -r requirements.txt

Checkpoints of backbones

All 11 pre-trained models in Table 1 of our paper can be found here. We use LightningFSL to train all backbones. For training the models, please refer to it.

Evaluate the simple channel-wise feature transformation

After obtaining pre-trained models and downloading needed datasets (see below), one can evaluate the simple channel-wise feature transformation. Exemplar commands for running the code can be found in eval.sh.

Download test-time datasets

Download any of the 19 test-time datasets in Table 1 of our paper for evaluation.

miniImageNet: Download and extract miniimagenet.tar, and use the test split as the root directory.
CUB: Download and extract cub.tar, and use the images/ folder as the root directory.
Textures: Download and extract dtd-r1.0.1.tar.gz, and use the images/ folder as the root directory.
Traffic Signs: Download and extract GTSRB_Final_Training_Images.zip, and use the Final_Training/Images/ folder as the root directory.
Aircraft: Download and extract fgvc-aircraft-2013b.tar.gz, and use the fgvc-aircraft-2013b/data folder as the root directory.
Omniglot: Download images_background.zip and images_evaluation.zip, then extract them into the same directory as the root directory.
VGG Flower: Download 102flowers.tgz and imagelabels.mat, then extract 102flowers.tgz, and put them into the same directory as the root directory.
MSCOCO: Download val2017.zip and annotations_trainval2017.zip, then extract them into the same directory as the root directory.
Quick Draw: Download and extract quickdraw.zip, and use the resultant folder as the root directory.
Fungi: Download and extract fungi_train_val.tgz, and use the images/ folder as the root directory.
Plant Disease: Download from kaggle, and use the train split as the root directory.
ISIC: Download ISIC2018_Task3_Training_Input.zip and ISIC2018_Task3_Training_GroundTruth.zip, then extract them into the same directory as the root directory.
EuroSAT: Dowaload and extract EuroSAT.zip, and use the resultant folder as the root directory.
ChestX: Download from kaggle, and use the resultant folder as the root directory.
Real: Download and extract real.zip, and use the resultant folder as the root directory.
Sketch: Download and extract sketch.zip, and use the resultant folder as the root directory.
Infograph: Download and extract infograph.zip, and use the resultant folder as the root directory.
Painting: Download and extract painting.zip, and use the resultant folder as the root directory.
Clipart: Download and extract clipart.zip, and use the resultant folder as the root directory.

Citation

If you find our work useful in your research please consider citing:

@inproceedings{
Luo2022Channel,
title={Channel Importance Matters in Few-Shot Image Classification},
author={Luo, Xu and Xu, Jing and Xu, Zenglin},
booktitle={International Conference on Machine Learning},
year={2022},
}