Middle school teachers’ implementation and perceptions of automated writing evaluation
Date
2024-10-24
Journal Title
Journal ISSN
Volume Title
Publisher
Computers and Education Open
Abstract
Despite research supporting the efficacy of Automated Writing Evaluation (AWE) in improving writing outcomes, inconsistent implementation by teachers raises concerns about the efficacy of these systems in practice. However, little is known about what factors influence teachers’ implementation and perceptions of AWE. This study examined the relationship between teachers’ implementation and perceptions of the MI Write AWE system, seeking to identify actionable factors that could enhance AWE implementation and acceptance in the future. A mixed-methods design was utilized, combining quantitative analysis of usage logs and survey data with qualitative insights from focus groups and interviews with 19 teachers who participated in a randomized controlled trial (RCT) testing the efficacy of MI Write on students’ writing outcomes. Quantitative data were subjected to descriptive and non-parametric statistical analyses, while qualitative data underwent a deductive coding process, offering an integrated view of MI Write's use and educators’ perceptions. Teachers implemented MI Write variably and not to the extent expected of them within the RCT, but they did report generally positive attitudes towards MI Write. Findings indicated that positive perceptions of system usability and usefulness may be insufficient to promote effective implementation. Instead, ecological factors such as curricular alignment and the challenge of incorporating AWE into existing workload, administrative support, and broader social and educational policy appeared as factors influencing implementation. Findings emphasize that teachers’ implementation and perceptions of AWE are dependent on a range of contextual elements beyond mere system functionality, suggesting that successful adoption requires addressing broader ecological considerations.
Description
This article was originally published in Computers and Education Open. The version of record is available at: https://doi.org/10.1016/j.caeo.2024.100231.
© 2024 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license ( http://creativecommons.org/licenses/by/4.0/).
Keywords
automated writing evaluation (AWE), automated feedback, writing, perceptions, implementation
Citation
Wilson, Joshua, Amanda Delgado, Corey Palermo, Tania M. Cruz Cordero, Matthew C. Myers, Halley Eacker, Andrew Potter, Jessica Coles, and Saimou Zhang. “Middle School Teachers’ Implementation and Perceptions of Automated Writing Evaluation.” Computers and Education Open 7 (December 2024): 100231. https://doi.org/10.1016/j.caeo.2024.100231.