Quickstart | Documentation | Unity Asset Store | Unity Forums Thread | License

The Geometry Sequence Player is a package for the Unity game engine which enables playback of large geometry sequences inside of Unity. The package can be used to playback pointcloud, mesh or textured mesh sequences.

It is available on either Github (for non-commercial use) or the Unity Asset Store (for commercial use):

What is a geometry sequence?

In a geometry sequence, each frame consists out of an individual mesh or pointcloud, which is shown at short Intervalls to create the illusion of an animation, kind of like a three dimensional flipbook. As each frame is independent from the next one, no limitations apply to your animation, unlike for example skinned meshes or blendshapes, where mesh topology can’t be changed. This applies for example to:

Pre-Rendered fluid caches
Baked Physiscs simulations
Animated Procedural meshes
4D scans with per frame textures
Volumetric videos
Pointclouds captured by RGBD sensors (Kinect, Realsense, Lidar)
Data visualisations

Features

• Can import sequences from almost any source, with an included converter that covers most standard file formats.
• Easy setup of sequences with an easy-to-use Editor UI and playback controls
• Integrated into the Unity Timeline
• Granular playback control for interactive experiences is provided through a scripting API
• Sequences don’t need to be loaded into your Unity project or even the RAM, everything is streamed at runtime from disk
• Highly efficient playback system that allows you to stream millions of points/polygons per second, even on lower end hardware

The package

Originally, this package was created to facilitate the playback of volumetric videos from LiveScan3D, captured with Depth sensors such as the Azure Kinect or Intel realsense. But I realized that it could also be useful for many more applications, such as getting fluid and physics simulations from Blender into Unity!

I have used the package intensively in many projects myself, so it’s well proven and tested. The package has been released on Github as open-source code for quite a while already, but to make the development and support a bit more sustainable, I now decided to also offer the package on the Unity Asset Store.
If you want to use the package commercially, or if you want to support our development, you can buy it there! Thank you very much for your support

Read Article on Frontiers (Open Acces)
Download PDF

Abstract

Introduction

Volumetric video production in commercial studios is predominantly produced using a multi-view stereo process that relies on a high two-digit number of cameras to capture a scene. Due to the hardware requirements and associated processing costs, this workflow is resource-intensive and expensive, making it unattainable for creators and researchers with smaller budgets. Low-cost volumetric video systems using RGBD cameras offer an affordable alternative. As these small, mobile systems are a relatively new technology, the available software applications vary in terms of workflow and image quality. In this paper we provide an overview of the technical capabilities of sparse camera volumetric video capture applications and assess their visual fidelity and workflow.

Materials and methods

We selected volumetric video applications that are publicly available, support capture with multiple Microsoft Azure Kinect cameras and run on consumer-grade computer hardware. We compared the features, usability, and workflow of each application and benchmarked them in five different scenarios. Based on the benchmark footage, we analyzed spatial calibration accuracy, artifact occurrence and conducted a subjective perception study with 19 participants from a game design study program to assess the visual fidelity of the captures.

Results

We evaluated three applications, Depthkit Studio, LiveScan3D and VolumetricCapture. We found Depthkit Studio to provide the best experience for novel users, while LiveScan3D and VolumetricCapture require advanced technical knowledge to be operated. The footage captured by Depthkit Studio showed the least amount of artifacts by a larger margin, followed by LiveScan3D and VolumetricCapture. These findings were confirmed by the participants who preferred Depthkit Studio over LiveScan3D and VolumetricCapture.

Discussion

Based on the results, we recommend Depthkit Studio for the highest fidelity captures. LiveScan3D produces footage of only acceptable fidelity but is the only candidate that is available as open-source software. We therefore recommend it as a platform for research and experimentation. Due to the lower fidelity and high setup complexity, we recommend VolumetricCapture only for specific use-cases where its ability to handle a high number of sensors in a large capture volume is required.

Read article on IEEE
Download PDF

Abstract

This study presents a mixed reality training concept designed to enhance medical students’ acquisition of surgical knot-tying skills, a fundamental component of surgical training critical for effective wound closure and tissue healing. Utilizing a virtual reality headset with video passthrough functionality, the system provides adaptive visual instructions tailored to the user’s hand movements during the knot-tying process. A prototype was developed based on the concept, featuring three-dimensional videos in which virtual instructor hands demonstrate each step of the procedure. The training concept was derived from an iterative, user-centered process encompassing requirement analysis, prototype development, and evaluation. Key functionalities include the ability to display thread tension and tensile strength, dynamically adapt learning speed to the user’s progress, and deliver personalized feedback by visually augmenting the hands and fingers. Evaluation results indicate that spatial and tangible interactions facilitated by the mixed reality training prototype support the acquisition of practical skills, bridging the gap between digital and physical simulation training.