Detection and Synthesis of Full‐Body Environment Interactions for Virtual Humans

Abstract

We present a new methodology for enabling virtual humans to autonomously detect and perform complex full‐body interactions with their environments. Given a parameterized walking controller and a set of motion‐captured example interactions, our method is able to detect when interactions can occur and to coordinate the detected upper‐body interaction with the walking controller in order to achieve full‐body mobile interactions in similar situations. Our approach is based on learning spatial coordination features from the example motions and on associating body‐environment proximity information to the body configurations of each performed action. Body configurations become the input to a regression system, which in turn is able to generate new interactions for different situations in similar environments. The regression model is capable of selecting, encoding and replicating key spatial strategies with respect to body coordination and management of environment constraints as well as determining the correct moment in time and space for starting an interaction. As a result, we obtain an interactive controller able to detect and synthesize coordinated full‐body motions for a variety of complex interactions requiring body mobility. Our results achieve complex interactions, such as opening doors and drawing in a wide whiteboard. The presented approach introduces the concept of learning interaction coordination models that can be applied on top of any given walking controller. The obtained method is simple and flexible, it handles the detection of possible interactions and is suitable for real‐time applications.