Learning-based decision making for safe and scalable autonomous separation assurance

dc.contributor.advisor Peng Wei
dc.contributor.author Brittain, Marc
dc.contributor.department Aerospace Engineering
dc.date 2021-06-11T00:46:32.000
dc.date.accessioned 2021-08-14T06:32:36Z
dc.date.available 2021-08-14T06:32:36Z
dc.date.copyright Sat May 01 00:00:00 UTC 2021
dc.date.embargo 2021-03-24
dc.date.issued 2021-01-01
dc.description.abstract <p>With growing air traffic density in the national airspace and the introduction of new airspace operations such as urban air mobility (UAM) and urban traffic management (UTM), today's airspace operations are reaching a new level of complexity. In traditional airspace, human air traffic controllers face increased workload as a result of growing air traffic, resulting in a limitation in airspace capacity. To overcome this human cognitive limitation, automation tools provide a way to increase the sector capacity above human cognitive limits while reducing the workload for air traffic controllers, leading to a safer airspace environment.</p> <p>For the envisioned high-density airspace operations to become a reality, a scalable autonomous air traffic control system is required to comprehend the complex airspace, communicate with autonomous aircraft, and perform tactical decision making under uncertainty. In addition, the autonomous air traffic control system should be flexible enough to accommodate both traditional and new airspace operations.</p> <p>Separation assurance and conflict resolution is a key component of air traffic control (ATC). This task involves ensuring aircraft maintain safe separation requirements while also meeting required time of arrival (RTA) constraints at airspace metering fixes. In addition, when a potential loss of separation event is predicted, tactical maneuver advisories need to be prescribed by the ATC to the aircraft to resolve conflicts.</p> <p>In this dissertation, a suite of learning-based frameworks, as well as a new concept of operations (ConOps) of decentralized autonomous separation assurance and conflict resolution are introduced to accommodate the high-density, stochastic, and dynamic structured airspace that are flexible enough to be extended to low-altitude airspace operations.</p> <p>First, a new ConOps for separation assurance is proposed where the task is shifted from a centralized human ATC to a decentralized framework where each aircraft is equipped with autonomous self-separation. This allows for framework scalability that is invariant to the number of aircraft in the airspace.</p> <p>Second, a key component to enable autonomous separation assurance is data scalabilty and generalization under uncertainty. In this dissertation, the separation assurance task is formulated as a Markov Decision Process (MDP) and solved using deep multi-agent reinforcement learning. For high-density airspace environments, a novel intruder aircraft encoding technique is introduced that leverages attention networks to achieve data scalibility, without sacrificing important information from the environment. In addition, to make the framework more practical, we demonstrate how the framework is able to handle various levels of communication uncertainty in the environment. In this case, the surrounding aircraft must strategically adapt to the aircraft that lose communication to prevent loss of separation (LOS).</p> <p>Third, to accommodate a heterogeneous airspace environment with both autonomous and human pilots, a novel intention learning framework is introduced to learn the behavior of intruder aircraft without relying on communication. This allows the framework to handle environments where aircraft may have different objectives that are unknown to one another, such in the case of competing companies providing low-altitude package delivery services.</p> <p>To evaluate the performance of the deep multi-agent reinforcement learning frameworks, an extension to the open source air traffic control simulator BlueSky is developed to allow for computing resource scalability and environment parallelization, providing an environment that runs effectively on local workstations and cloud computing environments. This allows for many orders of magnitude more simulations to be run in parallel, significantly reducing the training time of the frameworks, while increasing performance.</p> <p>Fourth, air traffic controllers may advise route changes to aircraft to effectively sequence aircraft or minimize delay. To accommodate the time-sensitivity of these advisories, a novel hierarchical deep reinforcement learning framework is proposed capable of resolving conflicts between aircraft while also minimizing delay through route changes and speed advisories. This framework is demonstrated on the NASA Sector 33 air traffic control environment with promising performance for complex airspace environments.</p>
dc.format.mimetype application/pdf
dc.identifier archive/lib.dr.iastate.edu/etd/18463/
dc.identifier.articleid 9470
dc.identifier.contextkey 23293799
dc.identifier.doi https://doi.org/10.31274/etd-20210609-24
dc.identifier.s3bucket isulib-bepress-aws-west
dc.identifier.submissionpath etd/18463
dc.identifier.uri https://dr.lib.iastate.edu/handle/20.500.12876/EzR2Qkgz
dc.language.iso en
dc.source.bitstream archive/lib.dr.iastate.edu/etd/18463/Brittain_iastate_0097E_19346.pdf|||Fri Jan 14 21:42:22 UTC 2022
dc.subject.keywords Air Traffic Management
dc.subject.keywords Artificial Intelligence
dc.subject.keywords Reinforcement Learning
dc.subject.keywords Separation Assurance
dc.title Learning-based decision making for safe and scalable autonomous separation assurance
dc.type article
dc.type.genre dissertation
dspace.entity.type Publication
relation.isOrgUnitOfPublication 047b23ca-7bd7-4194-b084-c4181d33d95d
thesis.degree.discipline Aerospace Engineering
thesis.degree.level dissertation
thesis.degree.name Doctor of Philosophy
Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
12.24 MB
Adobe Portable Document Format