Abstract: We present a pose adaptive few-shot learning procedure and a two-stage data interpolation regularization, termed Pose Adaptive Dual Mixup (PADMix), for single-image 3D reconstruction. While augmentations via interpolating feature-label pairs are effective in classification tasks, they fall short in shape predictions potentially due to inconsistencies between interpolated products of two images and volumes when rendering viewpoints are unknown. PADMix targets this issue with two sets of mixup procedures performed sequentially. We first perform an input mixup which, combined with a pose adaptive learning procedure, is helpful in learning 2D feature extraction and pose adaptive latent encoding. The stagewise training allows us to build upon the pose invariant representations to perform a follow-up latent mixup under one-to-one correspondences between features and ground-truth volumes. PADMix significantly outperforms previous literature on few-shot settings over the ShapeNet dataset and sets new benchmarks on the more challenging real-world Pix3D dataset.
Pose Adaptive Dual Mixup for Few-Shot Single-View 3D Reconstruction
This repository provides the source code for the paper Pose Adaptive Dual Mixup for Few-Shot Single-View 3D Reconstruction published in AAAI-22. The implementation is on ShapeNet.
Cite this work
@inproceedings{padmix,
title={Pose Adaptive Dual Mixup for Few-Shot Single-View 3D Reconstruction},
author={Cheng, Ta-Ying and
Yang, Hsuan-Ru and
Trigoni, Niki and
Chen, Hwann-Tzong and
Liu, Tyng-Luh},
booktitle={AAAI},
year={2022}
}
Prerequisites
Clone the Code Repository
git clone https://github.com/ttchengab/PADMix.git
Datasets
The ShapeNet dataset is available below:
- ShapeNet rendering images: http://cvgl.stanford.edu/data2/ShapeNetRendering.tgz
- ShapeNet voxelized models: http://cvgl.stanford.edu/data2/ShapeNetVox32.tgz
Get Started
Create two directories, one for saving templates, and the other for saving checkpoints:
mkdir template
mkdir ckpts_fewshot
To generate the priors, use the following command:
python saveTV.py
To train the ground truth autoencoder, use the following command:
python fewshot_AE.py
To train with Input Mixup, use the following command:
python fewshot_mixup_triplet.py
To train with Latent Mixup, use the following command:
python fewshot_latent_mixup.py
To evaluate the IoU, use the following command:
python fewshot_eval.py
Pretrained Models
The pretrained model under the one-shot setting on ShapeNet is available here.
