With growing air traffic density in the national airspace and the introduction of new airspace operations such as urban air mobility (UAM) and urban traffic management (UTM), today's airspace operations are reaching a new level of complexity. In traditional airspace, human air traffic controllers face increased workload as a result of growing air traffic, resulting in a limitation in airspace capacity. To overcome this human cognitive limitation, automation tools provide a way to increase the sector capacity above human cognitive limits while reducing the workload for air traffic controllers, leading to a safer airspace environment.

For the envisioned high-density airspace operations to become a reality, a scalable autonomous air traffic control system is required to comprehend the complex airspace, communicate with autonomous aircraft, and perform tactical decision making under uncertainty. In addition, the autonomous air traffic control system should be flexible enough to accommodate both traditional and new airspace operations.

Separation assurance and conflict resolution is a key component of air traffic control (ATC). This task involves ensuring aircraft maintain safe separation requirements while also meeting required time of arrival (RTA) constraints at airspace metering fixes. In addition, when a potential loss of separation event is predicted, tactical maneuver advisories need to be prescribed by the ATC to the aircraft to resolve conflicts.

In this dissertation, a suite of learning-based frameworks, as well as a new concept of operations (ConOps) of decentralized autonomous separation assurance and conflict resolution are introduced to accommodate the high-density, stochastic, and dynamic structured airspace that are flexible enough to be extended to low-altitude airspace operations.

First, a new ConOps for separation assurance is proposed where the task is shifted from a centralized human ATC to a decentralized framework where each aircraft is equipped with autonomous self-separation. This allows for framework scalability that is invariant to the number of aircraft in the airspace.

Second, a key component to enable autonomous separation assurance is data scalabilty and generalization under uncertainty. In this dissertation, the separation assurance task is formulated as a Markov Decision Process (MDP) and solved using deep multi-agent reinforcement learning. For high-density airspace environments, a novel intruder aircraft encoding technique is introduced that leverages attention networks to achieve data scalibility, without sacrificing important information from the environment. In addition, to make the framework more practical, we demonstrate how the framework is able to handle various levels of communication uncertainty in the environment. In this case, the surrounding aircraft must strategically adapt to the aircraft that lose communication to prevent loss of separation (LOS).

Third, to accommodate a heterogeneous airspace environment with both autonomous and human pilots, a novel intention learning framework is introduced to learn the behavior of intruder aircraft without relying on communication. This allows the framework to handle environments where aircraft may have different objectives that are unknown to one another, such in the case of competing companies providing low-altitude package delivery services.

To evaluate the performance of the deep multi-agent reinforcement learning frameworks, an extension to the open source air traffic control simulator BlueSky is developed to allow for computing resource scalability and environment parallelization, providing an environment that runs effectively on local workstations and cloud computing environments. This allows for many orders of magnitude more simulations to be run in parallel, significantly reducing the training time of the frameworks, while increasing performance.

Fourth, air traffic controllers may advise route changes to aircraft to effectively sequence aircraft or minimize delay. To accommodate the time-sensitivity of these advisories, a novel hierarchical deep reinforcement learning framework is proposed capable of resolving conflicts between aircraft while also minimizing delay through route changes and speed advisories. This framework is demonstrated on the NASA Sector 33 air traffic control environment with promising performance for complex airspace environments.