Creating a flexible LVC architecture for mixed reality training of the dismounted warfighter

Pollock, Brice
Major Professor
Eliot Winer
Committee Member
Journal Title
Journal ISSN
Volume Title
Research Projects
Organizational Units
Journal Issue
Electrical and Computer Engineering

Physical military training within military operations in urban terrain (MOUT) environments provides a realistic experience, albeit at high cost and limited scenario flexibility. Alternatively, training within serious games, often from a laptop, provides a low cost, highly flexible platform, but lacks sufficient realism and engagement for some applications. Live, virtual, and constructive (LVC) systems attempt to combine these two and capitalize on their strengths for joint forces training. However, current LVC training environments for the dismounted warfighter often are too small for realistic squad-sized training, constructed statically without ability to reconfigure quickly into new scenarios, are developed as standalone systems dependent on specific communication protocols, and do not enable realistic interaction between LVC entities.

In response to these challenges, a rapidly reconfigurable LVC training system was developed at Iowa State University, known as the Veldt. The physical Veldt environment consists of a 44`x60' room with modular walls configured in unique layouts for different training scenarios. These configurations contain doorways, windows, alleys and other openings, which may contain displays rendering the virtual environment for seamless integration between the physical and virtual worlds. A tracking system gathers position and orientation information on trainees, weapons and other objects and a clustered game engine then uses this information to create virtual representations of the trainees in the virtual world. This information is sent through a communication server which distributes it to other connected components such as game engines and simulations which populate the virtual world with live and constructive entities.

This thesis presents solutions to two key challenges in creation of the Veldt: 1) how to correlate all physical and virtual worlds for seamless interaction regardless of location and 2) how to design a network architecture that is easily extendable and can accommodate multiple protocol types. The correlation of physical and virtual worlds is necessary for entities, their models, and terrain. A central communication architecture became the first element of a solution by flexibly connecting entities' location, orientation, fire and other information without requiring individual connections between all components.

To enable appropriate collaboration between LVC trainees within the system, models must be visually indistinguishable regardless of interaction medium. However, most game engine and simulations contain separate, sometimes proprietary, model databases. A model-matching approach was applied to overcome this challenge, requiring only minor configuration of connected components for a set of common models common to all the components' databases. This approach resulted in a less extensive, non-identical common database, but is more easily scalable and requires less resources over other methods.

Terrain correlation is required to prevent issues with collaboration and fair fight between distributed LVC entities, where improper terrain correlation could create an unrealistic training environment. Similar to model database correlation, game engine and simulation systems typically contain separate, sometimes proprietary terrains and terrain formats. Because utilizing separate tools to convert from a single source into different formats often produces non-identical terrains, a single procedural terrain modeling framework was created and implemented for the Veldt system.

The solution to the second challenge of creating a protocol independent network architecture was achieved by processing the entity information flowing through a central communication server. With this design, the communication server receives information from one component in its native protocol, converts that into a world state, and then for all interested components, converts the world state into a component's native protocol and sends the information. Therefore, the communication server only requires packing and unpacking methods to and from a world state to easily extend the network architecture to include other protocols.

These methods were first evaluated within a user study conducted by the Research Institute for Studies in Education (RISE) at Iowa State University. The study results found high ratings of the system by participants on involvement, interaction, and immersion; indicating a near seamless physical-virtual correlation between environments. A interservice demonstration of the system involving many distributed components and multiple live, virtual, and constructive entities provided further evaluation. The successfulness of this demonstration, involving collaboration between participants in live and virtual environments, further proved the successful correlation of the physical and virtual worlds. In addition, the demonstration proved success of the protocol independent network architecture, as the scenario ran in real-time with negligible latency and with two differing protocols types.