Learning-based decision making for safe and scalable autonomous separation assurance

Thumbnail Image
Brittain, Marc
Major Professor
Peng Wei
Committee Member
Journal Title
Journal ISSN
Volume Title
Research Projects
Organizational Units
Organizational Unit
Aerospace Engineering

The Department of Aerospace Engineering seeks to instruct the design, analysis, testing, and operation of vehicles which operate in air, water, or space, including studies of aerodynamics, structure mechanics, propulsion, and the like.

The Department of Aerospace Engineering was organized as the Department of Aeronautical Engineering in 1942. Its name was changed to the Department of Aerospace Engineering in 1961. In 1990, the department absorbed the Department of Engineering Science and Mechanics and became the Department of Aerospace Engineering and Engineering Mechanics. In 2003 the name was changed back to the Department of Aerospace Engineering.

Dates of Existence

Historical Names

  • Department of Aerospace Engineering and Engineering Mechanics (1990-2003)

Related Units

Journal Issue
Is Version Of

With growing air traffic density in the national airspace and the introduction of new airspace operations such as urban air mobility (UAM) and urban traffic management (UTM), today's airspace operations are reaching a new level of complexity. In traditional airspace, human air traffic controllers face increased workload as a result of growing air traffic, resulting in a limitation in airspace capacity. To overcome this human cognitive limitation, automation tools provide a way to increase the sector capacity above human cognitive limits while reducing the workload for air traffic controllers, leading to a safer airspace environment.

For the envisioned high-density airspace operations to become a reality, a scalable autonomous air traffic control system is required to comprehend the complex airspace, communicate with autonomous aircraft, and perform tactical decision making under uncertainty. In addition, the autonomous air traffic control system should be flexible enough to accommodate both traditional and new airspace operations.

Separation assurance and conflict resolution is a key component of air traffic control (ATC). This task involves ensuring aircraft maintain safe separation requirements while also meeting required time of arrival (RTA) constraints at airspace metering fixes. In addition, when a potential loss of separation event is predicted, tactical maneuver advisories need to be prescribed by the ATC to the aircraft to resolve conflicts.

In this dissertation, a suite of learning-based frameworks, as well as a new concept of operations (ConOps) of decentralized autonomous separation assurance and conflict resolution are introduced to accommodate the high-density, stochastic, and dynamic structured airspace that are flexible enough to be extended to low-altitude airspace operations.

First, a new ConOps for separation assurance is proposed where the task is shifted from a centralized human ATC to a decentralized framework where each aircraft is equipped with autonomous self-separation. This allows for framework scalability that is invariant to the number of aircraft in the airspace.

Second, a key component to enable autonomous separation assurance is data scalabilty and generalization under uncertainty. In this dissertation, the separation assurance task is formulated as a Markov Decision Process (MDP) and solved using deep multi-agent reinforcement learning. For high-density airspace environments, a novel intruder aircraft encoding technique is introduced that leverages attention networks to achieve data scalibility, without sacrificing important information from the environment. In addition, to make the framework more practical, we demonstrate how the framework is able to handle various levels of communication uncertainty in the environment. In this case, the surrounding aircraft must strategically adapt to the aircraft that lose communication to prevent loss of separation (LOS).

Third, to accommodate a heterogeneous airspace environment with both autonomous and human pilots, a novel intention learning framework is introduced to learn the behavior of intruder aircraft without relying on communication. This allows the framework to handle environments where aircraft may have different objectives that are unknown to one another, such in the case of competing companies providing low-altitude package delivery services.

To evaluate the performance of the deep multi-agent reinforcement learning frameworks, an extension to the open source air traffic control simulator BlueSky is developed to allow for computing resource scalability and environment parallelization, providing an environment that runs effectively on local workstations and cloud computing environments. This allows for many orders of magnitude more simulations to be run in parallel, significantly reducing the training time of the frameworks, while increasing performance.

Fourth, air traffic controllers may advise route changes to aircraft to effectively sequence aircraft or minimize delay. To accommodate the time-sensitivity of these advisories, a novel hierarchical deep reinforcement learning framework is proposed capable of resolving conflicts between aircraft while also minimizing delay through route changes and speed advisories. This framework is demonstrated on the NASA Sector 33 air traffic control environment with promising performance for complex airspace environments.

Sat May 01 00:00:00 UTC 2021