Deep learning for decision making in cyber-physical systems

Thumbnail Image
Tan, Kai Liang
Major Professor
Sarkar, Soumik
Krishnamurthy, Adarsh
Ganapathysubramanian, Baskar
Bhattacharya, Sourabh
Sharma, Anuj
Committee Member
Journal Title
Journal ISSN
Volume Title
Research Projects
Organizational Units
Journal Issue
Is Version Of
Mechanical Engineering
Ever since the inception of the first computer, computer systems have helped enable the essential building blocks of the current modern society. Simple systems like a calculator to sophisticated systems like banking, transportation, and orbital launch systems. However, complex systems still have limitations to perform intelligent actions, such as exhaustively enumerating rule-based algorithms to cover every corner case. Recent progress of machine learning and deep learning has been accelerated with breakthroughs in solving traditionally time-consuming solutions with efficient and parallel computing architectures. This has enabled the possibility towards the fourth industrial revolution by transforming traditional manufacturing pipelines into cyber-physical systems. While deep learning has found success in the computer vision research community and commercial adoption, the potential impact for cyber-physical systems is likely to be monumental. This thesis highlights several possible pathways for decision-making cyber-physical systems, along with potential challenges that impede widespread adoption. These can be summarized as the following ideas: i) reliability, ii) generalization, and iii) reproducibility. In conventional engineering systems, these are important aspects required to be tested before allowing full-scale deployment to the public. While there are many years of development and experience in the engineering industry to develop reliable systems for widespread adoption, enabling cyber-physical systems for decision-making is still in its infancy stage, requiring addressing the challenges mentioned above to develop standards for the industry. To address these challenges, this thesis report investigates the reliability of a deep reinforcement learning system. Formally, the investigation focuses specifically on deep reinforcement learning case studies and their reliability. The first study dives into the robustness of deep reinforcement learning system used as a traffic controller for an intersection. The second study explores the reliability of a deep reinforcement learning-based controller by developing potential adversaries. Finally, the third study investigates robustifying deep reinforcement learning systems with the adversary model.
Subject Categories