Characterizing the changes in the evolution of deep learning models

Abstract

Modern software is increasingly incorporating a new kind of component, the deep learning (DL) model, to implement functionalities that have defied traditional programming. Like traditional components, these DL models also evolve. However, unlike traditional software, there is a gap in understanding and characterizing changes throughout the DL software evolution. To fill the gap, we studied 27K revisions from 969 top-rated DL models from GitHub, which have been developed using the three most popular libraries (i.e., TensorFlow, PyTorch, and Keras). We developed a taxonomy of changes made during the evolution of DL models. Also, we investigated the common changes and their intents quantitatively and qualitatively to understand the change dynamics of DL model evolution. Specifically, what are the common changes made to the model? How are these changes associated with different stages of the DL pipeline? How are change intents distributed in the context of DL applications? This thesis paves the way to characterize the changes in the evolution of DL models by answering those questions. It guides practitioners in effectively developing and maintaining DL software. Our findings reveal how library design and default parameter choices can affect the evolution of deep learning models and highlight the importance of identifying better change operators. We also identify several DL-specific quality issues addressed by the changes studied, highlighting the need for renewed attention from the refactoring community and tool developers.