Code for paper "RA-Depth: Resolution Adaptive Self-Supervised Monocular Depth Estimation"

Code for paper "RA-Depth: Resolution Adaptive Self-Supervised Monocular Depth Estimation"
Abstract: Existing self-supervised monocular depth estimation methods can get rid of expensive annotations and achieve promising results. However, these methods suffer from severe performance degradation when directly adopting a model trained on a fixed resolution to evaluate at other different resolutions. In this paper, we propose a resolution adaptive self-supervised monocular depth estimation method (RA-Depth) by learning the scale invariance of the scene depth. Specifically, we propose a simple yet efficient data augmentation method to generate images with arbitrary scales for the same scene. Then, we develop a dual high-resolution network that uses the multi-path encoder and decoder with dense interactions to aggregate multi-scale features for accurate depth inference. Finally, to explicitly learn the scale invariance of the scene depth, we formulate a cross-scale depth consistency loss on depth predictions with different scales. Extensive experiments on the KITTI, Make3D and NYU-V2 datasets demonstrate that RA-Depth not only achieves state-of-the-art performance, but also exhibits a good ability of resolution adaptation.

RA-Depth

This repo is for RA-Depth: Resolution Adaptive Self-Supervised Monocular Depth Estimation (arxiv), ECCV2022

If you think it is a useful work, please consider citing it.

@inproceedings{he_ra_depth,
  title={RA-Depth: Resolution Adaptive Self-Supervised Monocular Depth Estimation},
  author={Mu, He and Le, Hui and Yikai, Bian and Jian, Ren and Jin, Xie and Jian, Yang},
  booktitle={ECCV},
  year={2022}
}

Overview of RA-Depth

Basic results on KITTI dataset

Visualization Results of Resolution Adaptation

Training:

CUDA_VISIBLE_DEVICES=0 python train.py --model_name RA-Depth --scales 0 --png --log_dir models --data_path /datasets/Kitti/Kitti_raw_data

Testing:

CUDA_VISIBLE_DEVICES=0 python evaluate_depth.py --load_weights_folder /models/RA-Depth/ --eval_mono --height 192 --width 640 --scales 0 --data_path /datasets/Kitti/Kitti_raw_data --png

Infer a single depth map from a RGB:

CUDA_VISIBLE_DEVICES=0 python test_simple.py --image_path /test.png --model_name RA-Depth

Acknowledgement

  • The authors would like to thank Beibei Zhou and Kun Wang for their valuable suggestions and discussions.
  • Thank the authors for their superior works: monodepth2, DIFFNet.

Download Source Code

Download ZIP

Paper Preview

Aug 18, 2022