Reliability, Criterion Validity and Sensitivity to Growth: Extending Work on Two Algebra Progress Monitoring Measures

Thumbnail Image
Perkmen, Serkan
Olson, Jeannette
Major Professor
Committee Member
Journal Title
Journal ISSN
Volume Title
Research Projects
Organizational Units
Organizational Unit
Project AAIMS
Project AAIMS, funded from January 2004 through December 2007, was designed to achieve two objectives related to the teaching and learning of algebra for students with and without disabilities. First, we examined algebra curriculum, instruction, and assessment for students with and without disabilities and determine the extent to which they were aligned. Second, we developed algebra assessment tools that can be used for monitoring the progress of students with and without disabilities as they learn algebra. We then investigated the measures' reliability, validity, and sensitivity to growth. This website has information about the activities, research, and products associated with Project AAIMS. Our district partners include Fort Dodge Community School District, South Tama County Community School District, and Ballard Community School District.
Journal Issue
Is Version Of

This report is intended to extend previous work examining the reliability and validity of the Basic Skills and the Content-Analysis-Multiple Choice probes in two Iowa schools districts and to explore the use of the measures for monitoring student progress. One hundred thirty five students in grades nine to twelve participated in the study. Data were gathered from September 2005 to January 2006. Over four months of data collection, students completed two Basic Skills probes and two Content Analysis-Multiple Choice probes each month. We assessed the alternate form reliability and test-retest reliability of both types of probes. Our findings revealed that both types of probes possessed adequate levels of reliability with the Basic Skills probes demonstrating a higher level of reliability than the Content Analysis-Multiple Choice probes. To evaluate the validity of both types of probes, we gathered data from a variety of indicators of students’ proficiency in algebra including course grades, teachers’ evaluations of student proficiency and growth, and performance on standardized assessment instruments including the Iowa Test of Education Development (ITED) and the Iowa Algebra Aptitude Test (IAAT). We examined both concurrent and predictive validity. Our results revealed moderate correlations between both types of probes and other indicators of students’ proficiency in algebra. We examined whether the probes reflected student growth over time. The analyses revealed students grew .51 and .61 points each week on Basic Skills and Content Analysis- Multiple Choice probes, respectively. This result suggests that the Content Analysis-Multiple Choice probes may be slightly more sensitive in evaluating student growth. Finally, we were also interested in exploring whether students’ growth on the two types of probes was related to teachers’ rating of growth or gain scores on the IAAT. We found a small but significant correlation between teacher ratings of growth and both types of probes. No significant correlations existed between either type of probe and IAAT gain scores.


Technical Report #12