This dissertation reports on the evaluation of a computer-mediated collaborative animation description task, where synchronous corrective feedback (SCF) serves as the primary source for grammar learning. The evaluation was conducted in an English-as-a-Second-Language (ESL) writing program at a large research university in the United States. Using Chapelle’s (2001) CALL evaluation framework, this evaluation was carried out in three iterations as Study 1 (Spring 2017 to Spring 2018), Study 2 (Fall 2018 to Spring 2019), and Study 3 (Spring 2020) with 153 students in 17 classes.

Study 1 recruited 52 learners and evaluated language learning potential of a computer-mediated collaborative animation description task with a mixed-methods explanatory design. A treatment group and a comparison group attended a 50-minute lesson and underwent two collaborative animation description tasks hosted in Google Docs. The treatment group had their errors on definite and indefinite articles highlighted by the researcher during the tasks (i.e., indirect SCF), while the comparison group did not. Changes in their uses of the two articles were assessed via their individual performance on an animation description test before the collaborative writing task (pretest), immediately after (posttest), and two weeks after (delayed posttest). Two learners were invited to a stimulated recall session, expressing their perception about the task and SCF.

Serving as approximate replication of Study 1, Study 2 recruited a different sample of 50 learners and evaluated language learning potential, learner fit, and practicality of the computer-mediated collaborative animation description task where SCF was intended to facilitate their learning of the two articles. On top of what was investigated in Study 1, Study 2 examined the effects of the correct form provided in SCF (i.e., direct SCF), the effects of direct and indirect SCF on learners’ receptive knowledge operationalized in a sentence rewriting test, and its potential biased effects towards learners with a certain level of language analytical ability (LAA). In addition, Study 2 explored one teacher’s capability to provide SCF during a 50-minute lesson attended by approximately 20 students.

Serving as approximate replication of Study 1 and Study 2, Study 3 recruited another cohort of 51 learners and evaluated language learning potential, learner fit, and practicality of the computer-mediated collaborative animation description task where SCF created opportunities for learning of the two articles. While addressing research questions identical to those in Study 2, Study 3 modified the sentence rewriting test in Study 2 to measure learners’ receptive knowledge with greater precision. In short, Study 2 served as an approximate replication of Study 1, and Study 3 served as an approximate replication of Study 1 and Study 2.

Overall, findings from these three studies found pros and cons of SCF in this local context. Findings on language learning potential suggested that SCF in the collaborative writing task may result in small to medium effect sizes in a 50-minute lesson. However, its effectiveness may vary depending on the number of revision opportunities that individual learners encounter during the task. Findings on learner fit provided some evidence that SCF, depending on its content, may be effective only for learners with high LAA and that not providing SCF in a class hour may leave behind learners with low LAA. Furthermore, it was found that both high and low LAA learners could be disadvantaged when they do not encounter many revision opportunities. Findings on practicality suggested that SCF may not be delivered synchronously enough, potentially due to the class size and learners’ writing fluency in this context.

This replication project is significant in two senses. First, its replication speed far exceeds the current replication practices in L2 research. Study 2 was initiated one semester after Study 1, and Study 3 was initiated one academic year after Study 2. Although some of these studies are still unpublished, it would be reasonable to deem this replication as a stark contrast with the average length of 6.64 years between an initial study and a replication study in L2 research. Second, the present dissertation demonstrates that if implemented appropriately and purposefully, replication studies can be as meaningful as studies that are characterized as new, original, and innovative. Instead of drawing implications from a one-shot study, replication studies allow a researcher to admit flaws in the study design explicitly and iteratively address those flaws. Iterative evaluation of this sort could help researchers communicate with stakeholders with greater precision and confidence.