High Resolution 2D-/3D-Scanning and Deep Learning Segmentation for Digitization of Fragmented Wall Paintings

Abstract

The preservation and study of mural wall paintings often involve the collection of numerous fragments with unknown context. In this paper the authors present a case study involving a Roman wall painting discovered in 2013 at the European Cultural Park Bliesbruck-Reinheim. The objective of this work was to develop a semi-automated assistance system for the digitization, visualization, and digital repositioning of the Roman wall painting fragments. Therefore an easy-to-use scanner system was developed, that captures high-resolution 2D images of the front and back surfaces of the fragments, along with a height map of the backside. The contributing partners also developed a control and operating software for the scanner, as well as an automated software platform for visualization and repositioning of the digital fragments. The contributions of this paper include the introduction of a ML-based algorithm for background subtraction and segmentation of the front surface of the fragments. The technical realisation for fast and accurate image acquisition of the fragments, including sensor registration and highresolution capture, has been worked out. The system calibration process, hardware setup and data correction techniques are described in detail. Additionally, the challenges of pixel-wise image segmentation for distinguishing between background, inner contour (wall painting), and outer contour (fragment surface without painting) are discussed. Our proposed approach overcomes the limitations of training ML algorithms on high-resolution images by employing patch-wise training and leveraging small features instead of large-scale features. The digitization and segmentation process demonstrated promising results in preserving and reconstructing the roman wall painting fragments. The findings of this study contribute to the field of cultural heritage preservation and provide valuable insights as the developed equipment and methods are highly transferable to future digitization projects.