An experimental study of the effectiveness of Panorama as a maintenance tool

Abstract

<p>It is a well known fact that the amount of time and money spent debugging and maintaining software far outstrips the amount spent developing it. Reports suggest that a majority of this effort is related to information-seeking, concept-location, and software comprehension. In addition to being inherently complex activities, these problems are exacerbated due to the fact that the original developers are often unavailable to help with the maintenance activities. To perform a maintenance task programmers first form dynamic and static mental models of the program based on their prior knowledge of programming and on information available about the program. Aside from high level design documents, Javadoc-like documentation has been the primary means of obtaining such information about programs. There are two severe shortcomings of this form of documentation: a) as recognized by proponents of AOP, the code for many of the concerns of interest to the maintainer are scattered across different classes and files and b) there are a lot of classes and files for large systems leading to information overload and it is very hard and time-consuming to identify the few segments of code and documentation that are relevant for a specific maintenance activity.</p> <p>To address these problems, we have developed an Eclipse plug-in, Panorama, which allow master developers to document sections of code related to a specific developer concern and provides ways to make such information readily available to maintainers. This tool helps directly with information-seeking, concept location, and software comprehension activities and focuses maintainer's attention on the specific code and guidance necessary for a particular task. To validate the tool and our approach, we conducted an experimental study with 19 subjects assigning them several maintenance tasks on a fairly complex application (a multi-user Poker game). The study showed that the experimental group who had access to Panorama was able to complete more maintenance tasks in less time than the control group who did not have access to the tool. In addition, preliminary results indicate that the tool will have more impact on productivity of persons new to the application and programming environment than on experts.</p>