A software framework for initializing, running, and maintaining mixed reality environments

Abstract

<p>With the advent of cheaper mixed reality (MR) and virtual reality (VR) equipment, immersive simulation technology has seen use in many industries for training purposes. As this training becomes more complex and realistic, so does the hardware and software supporting it. For example, mixed reality military training may run on a clustered computer system and occur over a large area, requiring multiple 3D motion tracking systems to monitor user positions. Simulated weapons communicate wirelessly to a master simulation server, and multiple stereo video screens show trainees a view into a virtual world where their training will occur.</p> <p>In a complex scenario such as this, it is necessary to have tools to help with the tasks such as: 1) input parameter initialization (e.g., scenario, number of participants, etc.), 2) the real-time status of individual systems, 3) data flow and timing between systems, and 4) software and hardware malfunctions.</p> <p>In this article, the Mixed Reality ToolBox (MRT), developed to perform these operations, is presented. A review of current research and available products will be presented, with a focus on military training. Next, some of the issues that arise when combining physical, VR, and MR environments will be discussed. Finally, the components of MRT will be described, along with results showing MRT's effectiveness enabling different tasks. The ways in which MRT has greatly simplified tasks such as launching clustered applications, tracking down configuration problems, and dealing with 3D tracking in large mixed reality environments will also be discussed.</p>