MOTChallenge. The Multiple Object Tracking Benchmark.

In the recent past, the computer vision community has relied on several centralized benchmarks for performance evaluation of numerous tasks including object detection, pedestrian detection, 3D reconstruction, optical flow, single-object short-term tracking, and stereo estimation. Despite potential pitfalls of such benchmarks, they have proved to be extremely helpful to advance the state-of-the-art in the respective research fields. Interestingly, there has been rather limited work on the standardization of multiple target tracking evaluation. One of the few exceptions is the well-known PETS dataset, targeted primarily at surveillance applications. Even for this widely used benchmark, a common technique for presenting tracking results to date involves using different subsets of the available data, inconsistent model training and varying evaluation scripts. With this benchmark we would like to pave the way for a unified framework towards more meaningful quantification of multi-target tracking.

What do we provide?

We have created a framework for the fair evaluation of multiple people tracking algorithms. In this framework we provide:

• A large collection of datasets, some already in use and some new challenging sequences!
• Detections for all the sequences.
• A common evaluation tool providing several measures, from recall to precision to running time.
• An easy way to compare the performance of state-of-the-art tracking methods.
• Several challenges with subsets of data for specific tasks such as 3D tracking, pedestrian detection or multi-camera tracking.

We rely on the spirit of crowdsourcing, and we encourage researchers to submit their sequences to our benchmark, so the quality of multiple object tracking systems can keep increasing and tackling more challenging scenarios.

Challenges

• MOT15 2D/3D tracking: multiple object tracking in 2D or 3D on a collection of sequences that have been used in the community.

• MOT16 tracking: a fresh new set of more challenging 2D tracking sequences with precise annotations.

• MOT17 detection: you can now test you detector on our challenging real-world sequences!

• MOT17 tracking: the rules of the game have changed. Results have to be presented on 3 detection sets, to show the versatility of the tracking methods!

Workshops

We have organized several workshops in conjunction with the top vision conferences:

• 1st Workshop in Benchmarking Multi-Target Tracking, in conjunction with WACV 2015 in Hawaii, USA.

• 2nd Workshop in Benchmarking Multi-Target Tracking, in conjunction with ECCV 2016 in Amsterdam, Netherlands.

• 1st Joint BMTT-PETS Workshop on Tracking and Surveillance, in conjunction with CVPR 2017 in Hawaii, USA.