Teamwork assessment in cooperative video games: Exploring patterns, associations and validity
Date
2023-08
Authors
Farah, Yvonne
Major Professor
Advisor
Dorneich, Michael
Kelly, Jonathan
Gilbert, Stephen
Committee Member
Journal Title
Journal ISSN
Volume Title
Publisher
Altmetrics
Abstract
This thesis describes the process of developing a teamwork measurement system by investigating cooperative video game environments for teamwork assessment. The work aims to provide teamwork measurement and testbed design guidance through cooperative games.
Teams are relied on in a variety of fields, due to the increased complexity of tasks, and the need for interpersonal cognitive, behavioral, and affective processes. However, quantifying teamwork behaviors is a challenging process, due to their complexity and interpersonal nature. Teamwork testbeds offer task environments for teamwork research, measurement, and training. Testbeds require the ability to elicit observable teamwork behaviors that can provide evidence of the teamwork competencies. This study investigates cooperative games as teamwork testbeds. Cooperative video games have been shown to foster prosocial behaviors, communication, and cooperative activities. Additionally, commercial off the shelf video games have been used as simulations for teamwork training and assessment. However, work is needed to establish the relationship between cooperative features and teamwork behaviors and investigate the internal validity of these environments as assessment testbeds. This thesis addresses these challenges through three research questions: investigating the consistency of cooperative games in inducing teamwork behaviors, the associations between cooperative features and teamwork behaviors, and the teamwork behavioral differences between upper and lower performers. With a clearer association between cooperative features and teamwork behaviors, designers can target specific profiles of teamwork behaviors in their testbed design. A codebook of teamwork behaviors and cooperative features was developed and used to annotate gameplay footage from publicly accessible streaming platforms. The study annotated footage from 177 teams, across 18 video games categorized under four cooperative genres.
Consistency of cooperative genres in inducing teamwork competencies was supported through similar distributions of teamwork behaviors within genres, for most competencies. In contrast, competencies between genres were significantly different. Therefore, designers can use cooperative genres to target teamwork distributions of interest. Additionally, associations between cooperative features and teamwork behaviors were established within genres, by investigating the top three competencies associated with every cooperative feature. The associations serve as testbed design guidance, where cooperative features can be used to emphasize the top associated competencies. Finally, when comparing upper and lower performers based on their gaming outcomes, some significant frequency differences were observed, for situation assessment, and explicit and implicit coordination, indicating that the frequency of some behavioral markers affect performance in several genres.
This study investigated cooperative gaming genres to explore how and why they induce teamwork behaviors. Cooperative genres were found to be inducing different patterns of teamwork competencies, and therefore can be used by designers in teamwork testbed developments, to target specific teamwork profiles. Furthermore, cooperative features induced different proportions of teamwork behaviors, and therefore can be provided as tools to make certain behaviors observable and trackable. The findings support the validity of these environments for teamwork assessment and provide a range of assessment and design insights to develop and assess teamwork in cooperative testbeds, including the developed codebook, the consistent teamwork profiles, and the derived associations.
Series Number
Journal Issue
Is Version Of
Versions
Series
Academic or Administrative Unit
Type
thesis