Statistical and qualitative analysis of ChatGPT and human raters in preservice teachers' writing assessment

Basit Öğe Kaydı
dc.contributor.authorGulden, Bahadir
dc.contributor.authorBilge, Huzeyfe
dc.contributor.authorUysal, Pinar Kanik
dc.date.accessioned2026-02-28T12:18:11Z
dc.date.available2026-02-28T12:18:11Z
dc.date.issued2026
dc.departmentBayburt Üniversitesi
dc.description.abstractTeachers spend a significant amount of time providing feedback. This study compared expert and ChatGPT assessments and feedback on written texts to determine the suitability of AI for writing skill assessments that are time-consuming to assess and provide feedback. Three experts and ChatGPT graded 14 Turkish undergraduate students' assignments using rubric that included content, language use, vocabulary, organization, and mechanics, and justified their decisions. The study involved document review and triangulation, a qualitative design. In addition, an intraclass correlation coefficient was used to assess the consistency of the ChatGPT and the experts' scores. All feedback was qualitatively analyzed to identify the strengths and weaknesses of the experts and their similarities with ChatGPT. Experts and ChatGPT had moderate to weak consistency in the writing subscales, while good reliability was found in the total score. Experts excelled in 'explanatory feedback', 'interpretation' and 'experience', while ChatGPT excelled in 'automation and continuity' and 'data processing capacity'. Experts' weaknesses included 'limited time and energy' and 'comparison bias', while ChatGPT's weaknesses were 'ambiguous expressions' and 'repetition'. The study also found that experts and ChatGPT preferred to provide constructive and supportive feedback.
dc.identifier.doi10.21449/ijate.1678002
dc.identifier.endpage269
dc.identifier.issn2148-7456
dc.identifier.issue1
dc.identifier.startpage248
dc.identifier.urihttps://doi.org/10.21449/ijate.1678002
dc.identifier.urihttps://hdl.handle.net/20.500.12403/6149
dc.identifier.volume13
dc.identifier.wosWOS:001667972600001
dc.identifier.wosqualityQ3
dc.indekslendigikaynakWeb of Science
dc.language.isoen
dc.publisherIzzet Kara
dc.relation.ispartofInternational Journal of Assessment Tools in Education
dc.relation.publicationcategoryMakale - Uluslararası Hakemli Dergi - Kurum Öğretim Elemanı
dc.rightsinfo:eu-repo/semantics/openAccess
dc.snmzKA_WoS_20260218
dc.subjectArtificial Intelligence
dc.subjectChatGPT
dc.subjectWriting feedback
dc.subjectHuman-raters
dc.titleStatistical and qualitative analysis of ChatGPT and human raters in preservice teachers' writing assessment
dc.typeArticle

Dosyalar

Koleksiyon

WoS İndeksli Yayınlar Koleksiyonu