Accurate and Marker-less Head Tracking Using Depth Sensors

Abstract

Parameterized, high-fidelity 3D surface models can not only be used for rendering animations in the context of Computer Graphics (CG), but have become increasingly popular for analyzing data, and thus making these accessible to CG systems in an Analysis-by-Synthesis loop. In this paper, we utilize this concept for accurate head tracking by fitting a statistical 3D model to marker-less face data acquired with a low-cost depth sensor, and demonstrate its robustness in a challenging car driving scenario. We compute 3D head position and orientation with a mesh-based 3D shape matching algorithm that is independent of person identity and sensor type, and at the same time robust to facial expressions, speech, partial occlusion and illumination changes. Different strategies for obtaining the 3D face model are evaluated, trading off computational complexity and accuracy. Ground truth data for head pose are obtained from simultaneous marker-based tracking. Average tracking errors are below 6mm for head position and below 2.5 for head orientation, demonstrating the system's potential to be used as part of a non-intrusive head tracking system for use in Augmented Reality or driver assistance systems.