Incremental Reconstruction Approach for Telepresence or AR Applications

Abstract

This paper proposes an on-line incremental 3D reconstruction framework aimed at fulfilling the needs of telepresence or human machine interaction applications. The research presents a teleconference system that improves and induces the feeling that persons are in the presence of each other. A free viewpoint method, based on realistic user's appearances, is proposed to simulate a real face-to-face meeting. The contributions are: a new incremental version of Crust algorithm that enables incremental fusion of sensor data and a confidence-based method that automatically decides whether or not to integrate newly acquired data in the existing model based on measure uncertainty and novelty. To avoid the classical stereo vision reconstruction problems, the method bases on hybrid sensors to acquire simultaneous depth information and the corresponding texture image (e.g. kinect). This enables the alignment between acquired data and pre-adquired model by maximizing a criterion that is related with the matching between visual features and between acquired shapes. A mesh based representation enables the use of the surface topological geometric information during the data model integration process