Leveraging Classification Metrics for Quantitative System-Level Analysis with Temporal Logic Specifications
Date
2022-02
Authors
Major Professor
Advisor
Committee Member
Journal Title
Journal ISSN
Volume Title
Publisher
Institute of Electrical and Electronics Engineers
Abstract
In many autonomy applications, performance of perception algorithms is important for effective planning and control. In this paper, we introduce a framework for computing
the probability of satisfaction of formal system specifications given a confusion matrix, a statistical average performance measure for multi-class classification. We define the probability of satisfaction of a linear temporal logic formula given a specific initial state of the agent and true state of the environment. Then, we present an algorithm to construct a Markov chain that represents the system behavior under the composition of the perception and control components such that the probability of the temporal logic formula computed over the Markov chain is consistent with the probability that the temporal logic formula is satisfied by our system. We illustrate this approach on a simple example of a car with pedestrian on the sidewalk environment, and compute the probability of satisfaction of safety requirements for varying parameters of the vehicle. We also illustrate how satisfaction probability changes with varied precision and recall derived from the confusion matrix. Based on our results, we identify several opportunities for future work in developing quantitative system-level analysis that incorporates perception models.
Series Number
Journal Issue
Is Version Of
Versions
Series
Academic or Administrative Unit
Type
Presentation
Comments
This is a manuscript of a proceeding published as A. Badithela, T. Wongpiromsarn and R. M. Murray, "Leveraging Classification Metrics for Quantitative System-Level Analysis with Temporal Logic Specifications," 2021 60th IEEE Conference on Decision and Control (CDC), Austin, TX, USA, 2021, pp. 564-571, doi: 10.1109/CDC45484.2021.9683611.
Rights Statement
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.