Testing ratings of violent video games: how well do they measure up?

Center, Katherine
Major Professor
Craig A. Anderson
Committee Member
Journal Title
Journal ISSN
Volume Title
Research Projects
Organizational Units
Organizational Unit
Journal Issue

Decades of research shows a rise in the number of people playing video games, with the content of violent video games becoming increasingly realistic, interactive and unequivocal in depicting violent activity (Gitter, Ewell, Guadagno, Stillman, & Baumeister, 2013). Research also shows that exposure to video game violence increases aggression (for recent meta-analyses, see Anderson et al., 2010; Greitemeyer & Mügge, 2014). The combination of these two factors—growing numbers of players in addition to progressively violent games—appears to have important consequences.

The General Aggression Model demonstrates how factors in the immediate situation (e.g., having just played a violent video game) combine with factors that people bring with them to the situation (e.g. positive thoughts about using aggression) influence a person in the short term (changing a reaction). The General Aggression Model also describes how multiple aggressive episodes can lead to long term changes in aggression related person variables (Anderson & Bushman, 2002).

One key issue in the study of the effects of violent video games is how best to assess the violent content in these games. Three common methods of assessing the violent content in video games include: (1) participants’ rating of the amount of violence in a game or genre (Anderson & Dill, 2000); (2) official game ratings, such as ESRB ratings (Przybylski, Ryan, & Rigby, 2009); and (3) independent raters’ assessments of violent content in video games or genres (Weber, Ritterfeld, & Mathiak, 2006). Using participants’ ratings is direct and has been found to be valid (Busching, et al., 2013). Busching, et al. found that user ratings and expert ratings were both reliable and valid measures of the violent content in video games. However, there is still little consensus of what is the best practice when measuring the violent content in video games (Anderson et al., 2010). Therefore, this dissertation explored different methodologies to assess exposure to violent video games.

The current research utilized a cross-sectional study design, using preexisting data gathered as 9 separate studies. These studies were conducted at universities, elementary schools, and high schools as both laboratory experiments and in-class surveys.

The total sample included 4,746 participants; due to missing data, numbers do not add to 100%. The sample included 1175 children (385 girls, 600 boys; 8-17 years), 3525 adults (1729 women, 1685 men; 18-52 years), 2311 males, and 2132 females. Only 3 of the 9 studies assessed ethnicity; 942 participants in these 3 studies were Caucasian and 134 were other ethnicities. Participants were recruited from university (N=3548), high school (N=809), middle school (N=301) and elementary school (N=88) classes.

Study 1 addressed whether there are age related differences in perceptions of violence. Although it was hypothesized that children and adults may rate the violence in video games systematically different, in this analysis there were no differences between video game ratings of children and adults.

Study 2 was designed to test whether a novel operationalization of expert ratings predict users’ personal violence rating of video games. In study 2, exposure scores calculated using a novel operationalization of expert ratings—mean game-specific exposure—did predict users’ personal violence ratings of video games. Therefore, mean violence ratings of all participants who played a specific game may be a useful measure of the amount of violence in video games compared to personal violence ratings.

Study 3 assessed whether exposure to violent video games creates a systematic reduction in individual’s perceptions of the violent content of games; thereby reducing the usefulness of user violence ratings as a useful video game violence measure. In Study 3, differential exposure scores—video game violence exposure scores calculated without using user ratings of a particular game—did not reliably predict personal violence ratings of that video game. Differential exposure scores were not consistent in their ability to estimate the violent content across violent or even nonviolent games. Therefore, high exposure to violent video games does not lead to a systematic reduction in individuals’ violence ratings of the games that they play.

The final aim of this dissertation was to determine whether different operationalizations of expert ratings predict scores on aggression related personality measures. Across the 9 studies, participants completed a variety of scales, including the Buss Perry Aggression Questionnaire, the Narcissistic Personality Inventory, the Attitudes Toward Violence Scale, the Dissipation-Rumination Scale, and the National Youth Survey. All scales that were included in these analyses were measured in at least 3 studies.

In Study 4 there was no statistical advantage in using different operationalizations of violent video game exposure—mean game-specific exposure and mean person-game difference—compared to using the mean personal exposure score. Because there was no added benefit from using mean game-specific exposure or mean person-game difference, these two operationalizations are not recommended for use in future studies of violent video games. Exposure to video game violence, as measured by the mean personal exposure score, significantly predicted participants’ scores on 11 out of 13 of the aggressive personality measures. Scores on all of these measures moved in a more aggressive direction as exposure to violent video games increased.

Analyzing data in this dissertation satisfies methodological curiosity about how best to measure violent video game exposure. The current studies used new methods of combining player’s violence ratings across all players of a particular game. Busching, et al. (2013) concluded that player ratings and their operationalization of expert ratings were equally useful measures. However, these studies did not support the idea that there is a more accurate violence rating than personal violence rating. Furthermore, the ease of using personal violence ratings to assess the violent content of video games is far simpler than coding hundreds of games in order to calculate game-specific violence ratings. Busching, et al. (2013) compared the validity of using user ratings, expert ratings, official agency ratings of individual game titles as well as expert ratings of game genres and concluded that the best practices included using either expert ratings or player ratings. The results of the present studies support that conclusion.

In conclusion, using self-ratings of video game violence is an acceptable measurement technique. Personal violence rating is a valid, cheap, and fast way to measure the violence in video games. Therefore, the current author’s recommendation for future studies is to continue to use personal violence ratings as a measure of the violence in video games.