Code for paper "Towards Grand Unification of Object Tracking"

Code for paper "Towards Grand Unification of Object Tracking"
Abstract: We present a unified method, termed Unicorn, that can simultaneously solve four tracking problems (SOT, MOT, VOS, MOTS) with a single network using the same model parameters. Due to the fragmented definitions of the object tracking problem itself, most existing trackers are developed to address a single or part of tasks and overspecialize on the characteristics of specific tasks. By contrast, Unicorn provides a unified solution, adopting the same input, backbone, embedding, and head across all tracking tasks. For the first time, we accomplish the great unification of the tracking network architecture and learning paradigm. Unicorn performs on-par or better than its task-specific counterparts in 8 tracking datasets, including LaSOT, TrackingNet, MOT17, BDD100K, DAVIS16-17, MOTS20, and BDD100K MOTS. We believe that Unicorn will serve as a solid step towards the general vision model. Code is available at this https URL.

Unicorn 🦄 : Towards Grand Unification of Object Tracking

PWC PWC PWC PWC PWC PWC

Unicorn

This repository is the project page for the paper Towards Grand Unification of Object Tracking

Highlight

  • Unicorn is accepted to ECCV 2022 as an oral presentation!
  • Unicorn first demonstrates grand unification for four object-tracking tasks.
  • Unicorn achieves strong performance in eight tracking benchmarks.

Introduction

  • The object tracking field mainly consists of four sub-tasks: Single Object Tracking (SOT), Multiple Object Tracking (MOT), Video Object Segmentation (VOS), and Multi-Object Tracking and Segmentation (MOTS). Most previous approaches are developed for only one of or part of the sub-tasks.

  • For the first time, Unicorn accomplishes the great unification of the network architecture and the learning paradigm for four tracking tasks. Besides, Unicorn puts forwards new state-of-the-art performance on many challenging tracking benchmarks using the same model parameters.

This repository supports the following tasks:

Image-level

  • Object Detection
  • Instance Segmentation

Video-level

  • Single Object Tracking (SOT)
  • Multiple Object Tracking (MOT)
  • Video Object Segmentation (VOS)
  • Multi-Object Tracking and Segmentation (MOTS)

Demo

Unicorn conquers four tracking tasks (SOT, MOT, VOS, MOTS) using the same network with the same parameters.

video_demo_unicorn.mp4

Results

SOT

MOT (MOT17)

MOT (BDD100K)

VOS

MOTS (MOTS Challenge)

MOTS (BDD100K MOTS)

Getting started

  1. Installation: Please refer to install.md for more details.
  2. Data preparation: Please refer to data.md for more details.
  3. Training: Please refer to train.md for more details.
  4. Testing: Please refer to test.md for more details.
  5. Model zoo: Please refer to model_zoo.md for more details.

Citing Unicorn

If you find Unicorn useful in your research, please consider citing:

@inproceedings{unicorn,
  title={Towards Grand Unification of Object Tracking},
  author={Yan, Bin and Jiang, Yi and Sun, Peize and Wang, Dong and Yuan, Zehuan and Luo, Ping and Lu, Huchuan},
  booktitle={ECCV},
  year={2022}
}

Acknowledgments

  • Thanks YOLOX and CondInst for providing strong baseline for object detection and instance segmentation.
  • Thanks STARK and PyTracking for providing useful inference and evaluation toolkits for SOT and VOS.
  • Thanks ByteTrack, QDTrack and PCAN for providing useful data-processing scripts and evalution codes for MOT and MOTS.

Download Source Code

Download ZIP

Paper Preview

Jul 26, 2022