DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programs

dc.contributor.author Wardat, Mohammad
dc.contributor.author Dantas Cruz, Breno
dc.contributor.author Le, Wei
dc.contributor.department Computer Science
dc.date.accessioned 2021-12-17T13:17:28Z
dc.date.available 2021-12-17T13:17:28Z
dc.date.issued 2021-12-07
dc.description.abstract Deep Neural Networks (DNNs) are used in a wide variety of applications. However, as in any software application, DNN-based apps are afflicted with bugs. Previous work observed that DNN bug fix patterns are different from traditional bug fix patterns. Furthermore, those buggy models are non-trivial to diagnose and fix due to inexplicit errors with several options to fix them. To support developers in locating and fixing bugs, we propose DeepDiagnosis, a novel debugging approach that localizes the faults, reports error symptoms and suggests fixes for DNN programs. In the first phase, our technique monitors a training model, periodically checking for eight types of error conditions. Then, in case of problems, it reports messages containing sufficient information to perform actionable repairs to the model. In the evaluation, we thoroughly examine 444 models -53 real-world from GitHub and Stack Overflow, and 391 curated by AUTOTRAINER. DeepDiagnosis provides superior accuracy when compared to UMLUAT and DeepLocalize. Our technique is faster than AUTOTRAINER for fault localization. The results show that our approach can support additional types of models, while state-of-the-art was only able to handle classification ones. Our technique was able to report bugs that do not manifest as numerical errors during training. Also, it can provide actionable insights for fix whereas DeepLocalize can only report faults that lead to numerical errors during training. DeepDiagnosis manifests the best capabilities of fault detection, bug localization, and symptoms identification when compared to other approaches.
dc.description.comments This preprint is made available through arXiv:https://arxiv.org/abs/2112.04036. This work is licensed under the Creative Commons Attribution 4.0 License.
dc.identifier.uri https://dr.lib.iastate.edu/handle/20.500.12876/0zEy6n9z
dc.language.iso en
dc.publisher © Author(s) 2021
dc.source.uri https://arxiv.org/abs/2112.04036 *
dc.subject deep neural networks
dc.subject fault location
dc.subject debugging
dc.subject program analysis
dc.subject deep learning bugs
dc.subject.disciplines DegreeDisciplines::Physical Sciences and Mathematics::Computer Sciences::Software Engineering
dc.subject.disciplines DegreeDisciplines::Physical Sciences and Mathematics::Computer Sciences
dc.title DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programs
dc.type Preprint
dspace.entity.type Publication
relation.isAuthorOfPublication 4e3f4631-9a99-4a4d-ab81-491621e94031
relation.isOrgUnitOfPublication f7be4eb9-d1d0-4081-859b-b15cee251456
File
Original bundle
Now showing 1 - 1 of 1
Name:
2021-RajanH-DeepDiagnosisPreprint.pdf
Size:
1.21 MB
Format:
Adobe Portable Document Format
Description:
Collections