A mobile augmented reality thunderstorm training technique to enhance aviation weather theory knowledge curricula
Date
2023-08
Authors
Wang, Kexin
Major Professor
Advisor
Winer, Eliot
Dorneich, Michael
Krishnamurthy, Adarsh
Committee Member
Journal Title
Journal ISSN
Volume Title
Publisher
Altmetrics
Abstract
In the year 2020, 1,157 General Aviation (GA) accidents happened, of which 210 were fatal. Research suggests that some of these accidents are due to ineffective training in weather theory such as inconsistent program setup and reliance on 2D materials. Weather theory knowledge includes fundamental principles underlying various weather phenomena. This knowledge equips pilots with the ability to distinguish and effectively respond to diverse weather conditions they may encounter during flights. Inadequate training can lead to a poor level of competence in weather theory and impact pilot decision making. Research shows that most aviation programs focus on interpreting weather reports instead of teaching theory. In addition, current training usually occurs with 2D materials such as text, images, or video. For GA pilot students, it might be difficult to map 2D information into a 3D mental model and often resulting in discrepancies between course material and real-world situations.
Extended Reality (XR) technologies, encompassing virtual reality (VR), augmented reality (AR), and mixed reality (MR) offer capabilities to address these problems. High-fidelity simulators can provide aircraft movements according to a pilot’s input. However, such simulators are aircraft specific, costly to build and run, and difficult to access for pilot students. Compared to high-fidelity simulators, XR simulators are low-cost and can cover a range of aircraft. However, XR simulators often use head mounted displays (HMDs) to provide an immersive experience that still may not be affordable for pilot students. Despite an HMD-based simulator, a mobile-based XR system is even lower-cost and more accessible to students. Among the different XR systems running on mobile devices such as phone or tablet, mobile AR stands out due to its ability to provide both rotational and translational movement (6-DoF), while mobile VR is limited to rotational transformation only (3-DoF).
Mobile AR presents immersive content through commodity devices such as smartphones and tablets and can use 2D printed markers to ensure accurate registration of virtual content. Numerous studies have found that mobile AR systems can improve knowledge retention, reduce cognitive load, facilitate collaboration, increase motivation, and engage students in learning new skills. However, the rendering capabilities, camera tracking, and screen size on mobile devices may limit AR’s usability in complex phenomena and interactions. For example, weather topics such as thunderstorms have a lot of movement occurring inside the cloud, which can affect flight safety. Implementing a thunderstorm into an AR experience requires a realistic appearance, effective visual cues, and scenario activities to teach proper flight safety. To date, there has not been research on how to implement these into a mobile AR system. As a result, a new solution is needed. The research presented in this thesis studies the development and implementation of a thunderstorm simulation using AR deployed on mobile devices as a means to improve the teaching of GA pilot students.
To create a thunderstorm’s volumetric appearance, a particle system was implemented along with specialized shaders. Assessments and scenarios were then developed to provide specific experiences aimed at teaching pilot decision skills. To overcome limitations in the computational power of mobile devices, different viewing perspectives, specialized image targets, and user interfaces (UI) were also implemented to lower the rendering overhead, maintain stable tracking, and provide a smooth user experience. To better understand the application’s performance, a technical evaluation was performed to study the rendering and tracking performance of the application across different devices. For the purposes of this thesis, AR on a commodity device will be referred to as mobile AR.
Series Number
Journal Issue
Is Version Of
Versions
Series
Academic or Administrative Unit
Type
thesis