Developing and evaluating an approach to individualized, automated, process-focused feedback for university writing instruction

Abstract

Writing is an essential skill for success in many academic and professional settings. Despite receiving individualized feedback on their writing, many students struggle with writing in postsecondary education. This dissertation addresses this gap by focusing on the writing process—the moment-by-moment actions taken during writing—rather than the final product alone.

The first study employed thematic analysis of 300 session notes from a university writing center to understand the nature of feedback provided. This study contributes a novel characterization of feedback types and demonstrates the feasibility of using thematic analysis for session notes, paving the way for robust research methods in writing center contexts. The findings revealed that process-focused feedback is less prevalent than product-focused feedback in writing center sessions. These findings point to a mismatch between what occurs in writing center sessions and writing center expectations, which often center around improving students’ writing processes.

The second study utilized cluster analysis of keystroke and eye-tracking data across multiple writing sessions to investigate the stability of writing processes. Participants were 30 native-English-speaking students. The findings showed that writing processes are relatively stable across sessions; stability was substantially higher than what was predicted by chance. This study enhances the understanding of the stability of writing processes and introduces a granular view of the writing process through cluster analysis, enriching both theoretical and methodological aspects of writing process research and providing implications for process-focused interventions which rely on information from diagnostic sessions to develop appropriate behavioral interventions.

The third study employed design-based research for the development of “ProWrite,” an intelligent tutoring system utilizing keystroke and eye-tracking data to provide real-time feedback on the writing process. Five students engaged in three writing sessions using ProWrite. The study demonstrated the feasibility of ProWrite for providing real-time, process- and product-focused feedback, and there was evidence of short-term gains in writing quality for participants. This study shows the potential for developing an intelligent tutoring system for real-time, combined process- and product-focused feedback.

This dissertation makes significant contributions in understanding the nature of feedback provided in writing centers, the stability of writing processes across tasks, and the potential of intelligent tutoring systems for real-time feedback on the writing process. This research holds implications for writing pedagogy, aiming to revolutionize the delivery of writing instruction and support in postsecondary education.