Investigating the instructional effectiveness of TOEFL tutoring programs in an educational agency – A multi-method case study
Date
2023-08
Authors
Xiao, Meng
Major Professor
Advisor
Hagedorn, Linda S
Friedel, Janice N
Shelley, Mack C
Impecoven-Lind, Linda S
Thornton, Zoe M
Committee Member
Journal Title
Journal ISSN
Volume Title
Publisher
Altmetrics
Abstract
This dissertation study presents a case study of one of the largest educational agencies in China. The study investigated the instructional effectiveness of the test prep program in Test of English as a Foreign Language (TOEFL). The purpose of this study is threefold:1) to examine the educational effectiveness of a language proficiency test tutoring program provided by for-profit educational agents; 2) to explain and predict the cost of tutoring program outcomes with respect to point gain in the TOEFL test; and 3) to better understand the viewpoints of different types of clients, including students who took the tutoring sessions and parents who paid for the fee and oversaw the overall outcomes.
A theoretical perspective guides this study. Carroll's classic School Learning Model (1963) was adopted for the score change analyses to evaluate test-prep services' educational effectiveness and outcomes in increasing student scores. The theory states that educational outcomes are associated with time invested and exposure to learning and learning activities.
This study employs both a quantitative and qualitative approach. On the quantitative side, using a pre and post-test research design, students’ scores showed a significant increase after receiving the tutoring. For the cost analysis section, this study divided students into four groups based on their initial test scores: no gain, started low, started middle, and started high. The price per point was then calculated. In addition, in the qualitative portion, client satisfaction and opinions were presented using a blended framework of case studies and portraits. The clients reported overall satisfaction with the outcomes and considered it a worthy investment.
The findings of the study can provide agencies with best practices and further data collection ideas. This study is the first to apply the methodology that uses proportions of potential gain to measure students’ improvement in test scores. In addition, the suggestions to the international higher education institutions can help them understand the obstacles and struggles of international students and can further better serve this population. This study also provides insights to students and families and guides them to work with an agent wisely and better budget the education investment. Lastly, this study contributes to the literature on educational agency studies, score improvement measurement, and cost analyses.
Series Number
Journal Issue
Is Version Of
Versions
Series
Academic or Administrative Unit
Type
dissertation