Comparing Modalities to Communicate Movement Amplitude During Tool Manipulation in a Shared Learning Virtual Environment

Abstract

Shared immersive environments are used to teach technical skills and communicate relevant information. However, designing the appropriate interfaces and interactions to support this communication process remains an open issue. We explore using three modalities to communicate movement amplitude during tool manipulation tasks in a shared immersive environment. The haptic, visual, and verbal modalities were used separately to instruct a learner about the amplitude of the movements to perform in the 3D space. The user study comparing these modalities shows that instructions given through the visual modality permitted to decrease the distance estimation error. In contrast, the haptic modality helped the learners perform the task significantly faster. The verbal modality significantly increased the perceived sense of copresence but was the least preferred modality. This research contributes to understanding the importance of each modality when communicating spatial skills in a shared immersive environment. The results suggest that combining modalities could be the most appropriate way to transfer movement amplitude information to a learner by improving performance and user experience. These findings can enhance the design of immersive collaborative systems and open new perspectives for further research on the effectiveness of multimodal interaction to support learning technical skills in VR. Designed tools can be used in different fields, such as medical teaching applications.