A systemic functional perspective on automated writing evaluation: formative feedback on causal discourse

Thumbnail Image
Saricaoglu, Aysel
Major Professor
Carol Chapelle
Evgeny Chukharev-Hudilainen
Committee Member
Journal Title
Journal ISSN
Volume Title
Research Projects
Organizational Units
Organizational Unit

The Department of English seeks to provide all university students with the skills of effective communication and critical thinking, as well as imparting knowledge of literature, creative writing, linguistics, speech and technical communication to students within and outside of the department.

The Department of English and Speech was formed in 1939 from the merger of the Department of English and the Department of Public Speaking. In 1971 its name changed to the Department of English.

Dates of Existence

Historical Names

  • Department of English and Speech (1939-1971)

Related Units

Journal Issue
Is Version Of

Making explanations is a very important communicative function in academic literacy; several disciplines including science are dominated by causal explanations (Mohan & Slater, 2004; Slater, 2004; Wellington & Osborne, 2001). For academic success, students need to write about causes and effects well with the help of their instructors, which means that formative assessment of causal discourse is necessary (Slater & Mohan, 2010). However, manual evaluation of causal discourse is time-consuming and impractical for writing instructors. For this reason, automated evaluation of causal discourse, which current automated writing evaluation (AWE) systems cannot perform, is required. Addressing these needs, this dissertation aimed to develop an automated causal discourse evaluation tool (ACDET) and empirically evaluate learners’ causal discourse development with ACDET in academic writing classes.

ACDET was developed using three approaches: a functional linguistic approach, a hybrid natural language processing approach combining rule-based and statistical approaches, and a pedagogical approach. The linguistic approach helped identify causal discourse features by analyzing a small corpus of texts about causes and effects of economic events. ACDET detects seven types of causal discourse features and generates formative feedback based on them: causal conjunctions, causal adverbs, causal prepositions, causal verbs, causal adjectives, and causal nouns. The natural language processing approach allowed for assigning part-of-speech tags to sentences and words and creating hand-coded rules for the detection of causal discourse features. The pedagogical approach determined feedback features of ACDET, and it was informed by the theoretical perspectives of the Interaction Hypothesis and Systemic Functional Linguistics and findings of research on causal discourse development.

Causal discourse development with ACDET was empirically evaluated through a qualitative study in which four research questions investigated two criteria of computer-assisted language learning evaluation framework: language learning potential (i.e., focus on causal discourse form, interactional modifications, and causal discourse development) and focus on causal meaning. Participants of the study were 32 English as a second language learners who were students in two academic writing classes. Data consisted of pre- and post-tests, ACDET’s text-level feedback reports, cause-and-effect assignment drafts, screen capturing recordings, semi-structured interviews, and questionnaires.

The findings indicate language learning potential of ACDET: ACDET drew learners’ attention to causal discourse form and created opportunities for interactional modifications, however, resulted in limited causal discourse development. Findings also reveal that ACDET drew learners’ attention to causal meaning.

This study is an important attempt in the field of AWE to analyze meaning in written discourse automatically and provide causal discourse specific feedback. The fact that empirical evaluation of ACDET was based on process-oriented data revealing how students used ACDET in class is noteworthy. The findings of this study have important implications for the refinement of ACDET, the development of AWE systems, and research on causal discourse development.

Thu Jan 01 00:00:00 UTC 2015