Monocular Pose Reconstruction for an Augmented Reality Clothing System

Abstract

In this paper, we present an approach for realizing an augmented reality system for try-on of apparel. The core component of our system is a quick human pose estimation algorithm based on a single camera view only. Due to monocular input data, pose reconstruction may be ambiguous. We solve this problem by using a markered suit, though not relying on any specific marker layout. To recover 3D joint angles of the person using the system we use Relevance Vector Machine regression with image descriptors that include neighborhood configurations of visible colored markers and image gradient orientations. This novel combination of image descriptors results in a measurable improvement in reconstruction quality. We initialize and evaluate our algorithm with pose data acquired using a motion capture system. As the final step, we simulate a cloth draped around a virtual character adopting the estimated pose. Composing the original view and the rendered cloth creates the illusion of the user wearing virtual garments.