Qi Zhao

Ph.D. expected 2009
Advisor: Prof. Hai Tao
Department of Computer Engineering
School of Engineering
University of California, Santa Cruz

Office:	301 Engineering 2 Building
Phone:	(831)459-1248 (O)
Mobile:	(831)239-6516
Email:	zhaoqi AT soe DOT ucsc DOT edu
Address:	1156 High Street, Santa Cruz, CA 95064

Introduction

This work presents a real time video surveillance system which is capable of tracking multiple humans simultaneously. To better deal with various challenging issues such as occlusions, sharp motion changes and multi-person confusions, we propose an intelligent fusion framework where multiple cues are combined to seek the optimal objects state and more reliable cues have larger influences on the final decision. Further, part based human tracking provides a second-level information fusion in that parts with weak observability can be compensated by tracking other more visible ones, which demonstrates its effectiveness for highly articulated objects like humans.

Specifically, features we consider include motion, appearance and other local image information.
• Human motion is more complicated than motion of other objects like cars or faces. For humans, large accelerations or sudden changes in motion are common; and human articulation further aggravates the problem. Therefore human dynamic models are unreliable and used with caution in our method.
• Compared with motion, appearance information is more stable. People don’t tend to change appearance from frame to frame. We build an appearance model for each body part by clustering candidate body parts, and then use these models to measure appearance similarities for body parts in later frames.
• Other useful detection or tracking modules are also incorporated into our system. For example, we build a head detector based on convolutional neural network, which helps to detect and decompose humans in each frame. When the head detection module misses possible humans, a torso detection procedure is carried out to provide reliable observation data. Such information offers strong local image information, which greatly compensates the irregularity of the human motion.

Overview

Fig.1 System Diagram.

Experiments

(click the images to play videos)

Tracking in Low Resolution:

Tracking with Weak Foreground Mask Information:

Tracking with Occlusion:

References

Q. Zhao, J. Kang, H. Tao and W. Hua, "Part Based Human Tracking in a Multiple Cue Fusion Framework," in International Conference on Pattern Recognition (ICPR), vol. 1, pp. 450-455, Hongkong, China, August 2006. [pdf]