A Latent Dirichlet Allocation approach to understanding students’ perceptions of Automated Writing Evaluation
| Author(s) | Wilson, Joshua | |
| Author(s) | Zhang, Saimou | |
| Author(s) | Palermo, Corey | |
| Author(s) | Cordero, Tania Cruz | |
| Author(s) | Zhang, Fan | |
| Author(s) | Myers, Matthew C. | |
| Author(s) | Potter, Andrew | |
| Author(s) | Eacker, Halley | |
| Author(s) | Coles, Jessica | |
| Date Accessioned | 2024-12-10T16:06:26Z | |
| Date Available | 2024-12-10T16:06:26Z | |
| Publication Date | 2024-05-24 | |
| Description | This article was originally published in Computers and Education Open. The version of record is available at: https://doi.org/10.1016/j.caeo.2024.100194. © 2024 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). | |
| Abstract | Automated writing evaluation (AWE) has shown promise in enhancing students’ writing outcomes. However, further research is needed to understand how AWE is perceived by middle school students in the United States, as they have received less attention in this field. This study investigated U.S. middle school students’ perceptions of the MI Write AWE system. Students reported their perceptions of MI Write's usefulness using Likert-scale items and an open-ended survey question. We used Latent Dirichlet Allocation (LDA) to identify latent topics in students’ comments, followed by qualitative analysis to interpret the themes related to those topics. We then examined whether these themes differed among students who agreed or disagreed that MI Write was a useful learning tool. The LDA analysis revealed four latent topics: (1) students desire more in-depth feedback, (2) students desire an enhanced user experience, (3) students value MI Write as a learning tool but desire greater personalization, and (4) students desire increased fairness in automated scoring. The distribution of these topics varied based on students’ ratings of MI Write's usefulness, with Topic 1 more prevalent among students who generally did not find MI Write useful and Topic 3 more prominent among those who found MI Write useful. Our findings contribute to the enhancement and implementation of AWE systems, guide future AWE technology development, and highlight the efficacy of LDA in uncovering latent topics and patterns within textual data to explore students’ perspectives of AWE. | |
| Sponsor | This work was supported, in whole or in part, by the Bill & Melinda Gates Foundation [INV-006167]. Under the grant conditions of the Foundation, a Creative Commons Attribution 4.0 Generic License has already been assigned to the Author Accepted Manuscript version that might arise from this submission. The opinions expressed in this paper are those of the authors and do not represent the views of the Foundation, and no official endorsement by this agency should be inferred. Declaration of competing interest The following authors declare no conflicts of interest relevant to this work: Joshua Wilson, Saimou Zhang, Fan Zhang, Tania Cruz Cordero, Matthew C. Myers, Andrew Potter. The following authors are or were employed by Measurement Incorporated at the time the research was conducted: Corey Palermo, Halley Eacker, Jessica Coles. During the preparation of this work the authors used ChatGPT in order to revise portions of the text for clarity and succinctness. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication. | |
| Citation | Wilson, Joshua, Saimou Zhang, Corey Palermo, Tania Cruz Cordero, Fan Zhang, Matthew C. Myers, Andrew Potter, Halley Eacker, and Jessica Coles. “A Latent Dirichlet Allocation Approach to Understanding Students’ Perceptions of Automated Writing Evaluation.” Computers and Education Open 6 (June 2024): 100194. https://doi.org/10.1016/j.caeo.2024.100194. | |
| ISSN | 2666-5573 | |
| URL | https://udspace.udel.edu/handle/19716/35650 | |
| Language | en_US | |
| Publisher | Computers and Education Open | |
| dc.rights | Attribution 4.0 International | en |
| dc.rights.uri | http://creativecommons.org/licenses/by/4.0/ | |
| Keywords | automated writing evaluation | |
| Keywords | automated feedback | |
| Keywords | feedback | |
| Keywords | latent dirichlet allocation | |
| Keywords | LDA | |
| Keywords | perceptions | |
| Title | A Latent Dirichlet Allocation approach to understanding students’ perceptions of Automated Writing Evaluation | |
| Type | Article |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- A Latent Dirichlet Allocation approach to understanding students’ perceptions of Automated Writing Evaluation.pdf
- Size:
- 1.86 MB
- Format:
- Adobe Portable Document Format
- Description:
- Main article
License bundle
1 - 1 of 1
No Thumbnail Available
- Name:
- license.txt
- Size:
- 2.22 KB
- Format:
- Item-specific license agreed upon to submission
- Description: