Read Article on Frontiers (Open Acces)
Download PDF

Abstract

Introduction

Volumetric video production in commercial studios is predominantly produced using a multi-view stereo process that relies on a high two-digit number of cameras to capture a scene. Due to the hardware requirements and associated processing costs, this workflow is resource-intensive and expensive, making it unattainable for creators and researchers with smaller budgets. Low-cost volumetric video systems using RGBD cameras offer an affordable alternative. As these small, mobile systems are a relatively new technology, the available software applications vary in terms of workflow and image quality. In this paper we provide an overview of the technical capabilities of sparse camera volumetric video capture applications and assess their visual fidelity and workflow.

Materials and methods

We selected volumetric video applications that are publicly available, support capture with multiple Microsoft Azure Kinect cameras and run on consumer-grade computer hardware. We compared the features, usability, and workflow of each application and benchmarked them in five different scenarios. Based on the benchmark footage, we analyzed spatial calibration accuracy, artifact occurrence and conducted a subjective perception study with 19 participants from a game design study program to assess the visual fidelity of the captures.

Results

We evaluated three applications, Depthkit Studio, LiveScan3D and VolumetricCapture. We found Depthkit Studio to provide the best experience for novel users, while LiveScan3D and VolumetricCapture require advanced technical knowledge to be operated. The footage captured by Depthkit Studio showed the least amount of artifacts by a larger margin, followed by LiveScan3D and VolumetricCapture. These findings were confirmed by the participants who preferred Depthkit Studio over LiveScan3D and VolumetricCapture.

Discussion

Based on the results, we recommend Depthkit Studio for the highest fidelity captures. LiveScan3D produces footage of only acceptable fidelity but is the only candidate that is available as open-source software. We therefore recommend it as a platform for research and experimentation. Due to the lower fidelity and high setup complexity, we recommend VolumetricCapture only for specific use-cases where its ability to handle a high number of sensors in a large capture volume is required.

Read article on IEEE
Download PDF

Abstract

This study presents a mixed reality training concept designed to enhance medical students’ acquisition of surgical knot-tying skills, a fundamental component of surgical training critical for effective wound closure and tissue healing. Utilizing a virtual reality headset with video passthrough functionality, the system provides adaptive visual instructions tailored to the user’s hand movements during the knot-tying process. A prototype was developed based on the concept, featuring three-dimensional videos in which virtual instructor hands demonstrate each step of the procedure. The training concept was derived from an iterative, user-centered process encompassing requirement analysis, prototype development, and evaluation. Key functionalities include the ability to display thread tension and tensile strength, dynamically adapt learning speed to the user’s progress, and deliver personalized feedback by visually augmenting the hands and fingers. Evaluation results indicate that spatial and tangible interactions facilitated by the mixed reality training prototype support the acquisition of practical skills, bridging the gap between digital and physical simulation training.