The Application of Chat GPT in English Language Evaluation: A Systematic Literature Review
DOI:
https://doi.org/10.57125/FED.2024.09.25.13Keywords:
Systematic Literature Review, Chat GPT, English Language AssessmentAbstract
This comprehensive study of the literature investigated the use, advantages, and drawbacks of the Chat GPT in English language instruction. This article was constructed by using a limited search strategy for relevant databases like Google Scholar, Frontiersin, ResearchGate, Journal of Applied Learning and Teaching, and ScienceDirect using the keywords “Chat GPT” and “ELT Assessment.” The search is limited to articles published between 2019 and 2024, written in English, and containing “Chat GPT” and “English Language Assessment,” open-access, peer-reviewed, and relevant conference papers. This review examined 20 academic research papers on the use of the Chat GPT in English Language Teaching (ELT) assessment. The findings highlighted various benefits, including simplified assessment, customisation, near-real-time automation, removal of geographical restrictions, and cost-effectiveness. Nevertheless, ethical issues and restrictions were discussed, such as the responsible use of AI as a tool and the imbalance that AI can create in the integrity of an the assessment. The outcomes of this review could offer an understanding of the Chat GPT in terms of ELT assessment application and some new opportunities for future research and practice. The novelty of this study lies in the thorough literature review conducted by the author, encompassing a diverse array of research studies. The study elucidated various advantages and obstacles encountered by researchers during the assessment process. Furthermore, the researchers underscored the necessity of human validation for current generational artificial intelligence models in practical applications.
References
Ahmed, S. M. A. A., Taha, A. R. A., Hussain, S., & Hayat, A. (2023). Enhancing the teaching and learning of English for Specific Purposes (ESP) with ChatGPT. International Journal of Technology and Education Research, 1(03), 40–49. https://www.e-journal.citakonsultindo.or.id/index.php/IJETER/article/view/458
Akiba, D., & Fraboni, M. C. (2023). AI-supported academic advising: Exploring ChatGPT’s current state and future potential toward student empowerment. Education Sciences, 13(9). https://doi.org/10.3390/educsci13090885
Al-Amin, M., Shazed Ali, M., Salam, A., Khan, A., Ali, A., Ullah, A., Alam, N., & Chowdhury, S. K. (2024). History of generative Artificial Intelligence (AI) chatbots: Past, present, and future development. arXiv. https://doi.org/10.48550/arXiv.2402.05122
Alafnan, M. A., Dishari, S., Jovic, M., & Lomidze, K. (2023). ChatGPT as an educational tool: Opportunities, challenges, and recommendations for communication, business writing, and composition courses. Journal of Artificial Intelligence and Technology, 3(2), 60–68. https://doi.org/10.37965/jait.2023.0184
Algaraady, J., & Mahyoob, M. (2023). ChatGPT’s Capabilities in Spotting and Analyzing Writing Errors Experienced by EFL Learners. Arab World English Journal, 9, 3–17. https://doi.org/10.24093/awej/call9.1
Ali, J. K. M. (2023). Benefits and challenges of using ChatGPT: An exploratory study on English language program. University of Bisha Journal for Humanities, 2(2), 629–641. https://www.researchgate.net/publication/372310955
Amin, M. Y. M. (2023). AI and Chat GPT in language teaching: Enhancing EFL classroom support and transforming assessment techniques. International Journal of Higher Education Pedagogies, 4(4), 1–15. https://doi.org/10.33422/ijhep.v4i4.554
Athanassopoulos, S., Manoli, P., Gouvi, M., Lavidas, K., & Komis, V. (2023). The use of ChatGPT as a learning tool to improve foreign language writing in a multilingual and multicultural classroom. Advances in Mobile Learning Educational Research, 3(2), 818–824. https://doi.org/10.25082/amler.2023.02.009
Bahrini, A., Khamoshifar, M., Abbasimehr, H., Riggs, R. J., Esmaeili, M., Majdabadkohne, R. M., & Pasehvar, M. (2023). ChatGPT: Applications, opportunities, and threats. In 2023 Systems and Information Engineering Design Symposium (pp. 274–279). IEEE. https://doi.org/10.1109/SIEDS58326.2023.10137850
Fauzi, F., Tuhuteru, L., Sampe, F., Ausat, A., & Hatta, H. (2023). Analysing the role of ChatGPT in improving student productivity in higher education. Journal on Education, 5(4), 14886–14891. https://doi.org/10.31004/joe.v5i4.2563
Fitria, T. N. (2023). Artificial intelligence (AI) technology in OpenAI ChatGPT application: A review of ChatGPT in writing English essay. ELT Forum: Journal of English Language Teaching, 12(1), 44–58. https://doi.org/10.15294/elt.v12i1.64069
Flaagan, T. (2023). Using ChatGPT to generate computer science problem sets. International Journal on Integrating Technology in Education, 12(2), 19–29. https://doi.org/10.5121/ijite.2023.12202
Fontenelle-Tereshchuk, D. (2024). Academic writing and ChatGPT: Students transitioning into college in the shadow of the COVID-19 pandemic. Discover Education, 3(1), Article 6. https://doi.org/10.1007/s44217-023-00076-5
Gao, J., Galley, M., & Li, L. (2018). Neural approaches to conversational AI. arXiv. http://arxiv.org/abs/1809.08267
Ghafar, Z. N. (2023). ChatGPT: A new tool to improve teaching and evaluation of second and foreign languages a review of ChatGPT: The future of education. International Journal of Applied Research and Sustainable Sciences (IJARSS), 1(2), 73–86. https://doi.org/10.59890/ijarss.v1i2.392
Mogavi, R. H., Deng, C., Kim, J. J., Zhou, P., Kwon, Y. D., Metwally, A. H. S. ... Hui, P. (2024). ChatGPT in education: A blessing or a curse? A qualitative study exploring early adopters’ utilization and perceptions. Computers in Human Behavior: Artificial Humans, 2(1), Article 100027. https://doi.org/10.1016/j.chbah.2023.100027
Imran, M., & Almusharraf, N. (2023). Analyzing the role of ChatGPT as a writing assistant at higher education level: A systematic review of the literature. Contemporary Educational Technology, 15(4), Article ep464. https://doi.org/10.30935/cedtech/13605
Ivanashko, O., Kozak, A., Knysh, T., & Honchar, K. (2024). The role of artificial intelligence in shaping the future of education: Opportunities and challenges. Futurity Education, 4(1), 126–146. https://doi.org/10.57125/fed.2024.03.25.08
Japoshvili-Ghvinashvili, M., & Suleman, N. (2023). Assisting ELT teachers: Designing activities for the use of ChatGPT in teaching and learning. Pakistan Journal of Multidisciplinary Innovation, 2(1), 24–35. https://doi.org/10.59075/pjmi.v2i1.219
Jones, B. (2023). Breaking boundaries with generative AI: Good practice of unleashing the power of ChatGPT for inter- and transdisciplinary breakthroughs in the age of complexity. Journal of the European Honors Council, 6(2). https://doi.org/10.31378/jehc.177
Khan, K. S., Kunz, R., Kleijnen, J., & Antes, G. (2003). Five steps to conducting a systematic review. Journal of the Royal Society of Medicine, 96(3), 118–121. https://doi.org/10.1177/014107680309600304
Khondaker, M. T. I., Waheed, A., Nagoudi, E. M. B., & Abdul-Mageed, M. (2023). GPTAraEval: A comprehensive evaluation of ChatGPT on Arabic NLP. arXiv. https://doi.org/10.48550/arXiv.2305.14976
Koraishi, O. (2023). Teaching English in the age of AI: Embracing ChatGPT to optimize EFL materials and assessment. Language Education & Technology(LET Journal), 3(1), 55–72. https://langedutech.com/letjournal/index.php/let/article/view/48
Law, L. (2024). Application of generative artificial intelligence (GenAI) in language teaching and learning: A scoping literature review. Computers and Education Open, 6, Article 100174. https://doi.org/10.1016/j.caeo.2024.100174
Liu, Y. (2022). Artificial intelligence and its application in educational industry. Frontiers in Educational Research, 5(6), 44–47. https://doi.org/10.25236/fer.2022.050608
Lubis, A. H., Ayuningsih, A. S., & Ikaputri, S. R. (2024). A systematic literature review of ChatGPT utilization in education: Potentials and concerns. In Proceedings of the 7th International Conference on Language, Literature, Culture, and Education (ICOLLITE 2023) (pp. 38–44). Atlantis Press. https://doi.org/10.2991/978-94-6463-376-4_6
Mai, L., & Carson-Berndsen, J. (2023). Enhancing conversational quality in language learning chatbots: An evaluation of GPT4 for ASR error correction. arXiv. https://doi.org/10.48550/arXiv.2307.09744
Meniado, J. C. (2023). The impact of ChatGPT on English language teaching, learning, and assessment: A rapid review of literature. Arab World English Journal, 14(4), 3–18. https://doi.org/10.24093/awej/vol14no4.1
Mengist, W., Soromessa, T., & Legese, G. (2020). Ecosystem services research in mountainous regions: A systematic literature review on current knowledge and research gaps. Science of the Total Environment, 702, Article 134581. https://doi.org/10.1016/j.scitotenv.2019.134581
Mizumoto, A., & Eguchi, M. (2023). Exploring the potential of using an AI language model for automated essay scoring. Research Methods in Applied Linguistics, 2(2), 100050. https://doi.org/10.1016/j.rmal.2023.100050
Mohd Nawi, N. S., & Muhmad Nor, N. A. A. (2023). The challenges in the teaching of English literature. Journey: Journal of English Language and Pedagogy, 6(1), 130–147. https://doi.org/10.33503/journey.v6i1.2640
Monika, M., & Suganthan, C. (2024). A study on analyzing the role of Chatgpt in English acquisition among ESL learners during English language classroom. Bodhi International Journal of Research in Humanities, Arts and Science, 8(2), 75–84.
Nguyen, T. T. H. (2023). EFL teachers' perspectives toward the use of ChatGPT in writing classes: A case study at Van Lang University. International Journal of Language Instruction, 2(3), 1–47. https://doi.org/10.54855/ijli.23231
Nikolic, S., Daniel, S., Haque, R., Belkina, M., Hassan, G. M., Grundy, S. ... Sandison, C. (2023). ChatGPT versus engineering education assessment: A multidisciplinary and multi-institutional benchmarking and analysis of this generative artificial intelligence tool to investigate assessment integrity. European Journal of Engineering Education, 48(4), 559–614. https://doi.org/10.1080/03043797.2023.2213169
Ogunleye, B. (2024). Higher education assessment practice in the era of generative AI tools. Journal of Applied Learning & Teaching, 7(1), 46–56. https://doi.org/10.37074/jalt.2024.7.1.28
Park, H.-Y. (2023). Application of ChatGPT for an English learning platform. STEM Journal, 24(3), 30–48. https://doi.org/10.16875/stem.2023.24.3.30
Parker, L., Carter, C., Karakas, A., Loper, A. J., & Sokkar, A. (2024). Graduate instructors navigating the AI frontier: The role of ChatGPT in higher education. Computers and Education Open, 6, Article 100166. https://doi.org/10.1016/j.caeo.2024.100166
Qureshi, B. (2023). Exploring the use of ChatGPT as a tool for learning and assessment in undergraduate computer science curriculum: Opportunities and challenges. arXiv. https://doi.org/10.48550/arXiv.2304.11214
Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?. Journal of Applied Learning and Teaching, 6(1), 342–363. https://doi.org/10.37074/jalt.2023.6.1.9
Schmidt-Fajlik, R. (2023). ChatGPT as a grammar checker for Japanese English language learners: A comparison with Grammarly and ProWritingAid. AsiaCALL Online Journal, 14(1), 105–119. https://doi.org/10.54855/acoj.231417
Semeniuk, R., Holovnia, Y., Huda, O., & Savastru, N. (2024). Innovative methodologies and approaches to teaching with artificial intelligence in Ukrainian higher education. Futurity Education, 4(1), 24–52. https://doi.org/10.57125/fed.2024.03.25.02
Song, C., & Song, Y. (2023). Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students. Frontiers in Psychology, 14, Article 1260843. https://doi.org/10.3389/fpsyg.2023.1260843
Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J. ... Olson, C. B. (2024). Comparing the quality of human and ChatGPT feedback of students’ writing. Learning and Instruction, 91, Article 101894. https://doi.org/10.1016/j.learninstruc.2024.101894
Su, J., & Yang, W. (2023). Unlocking the power of ChatGPT: A framework for applying generative AI in education. ECNU Review of Education, 6(3), 355–366. https://doi.org/10.1177/20965311231168423
Tan, X. (2023). The impact of ChatGPT on education and future prospects. Highlights in Science, Engineering and Technology, 61, 138–143. https://doi.org/10.54097/hset.v61i.10285
Teng, F. (2023). Scientific writing, reviewing, and editing for open-access TESOL journals: The role of ChatGPT. International Journal of TESOL Studies, 5(1), 87–91. https://doi.org/10.58304/ijts.20230107
Trust, T., Whalen, J. & Mouza, C. (2023). Editorial: ChatGPT: Challenges, opportunities, and implications for teacher education. Contemporary Issues in Technology and Teacher Education, 23(1), 1–23. https://www.learntechlib.org/primary/p/222408/
Uchiyama, S., Umemura, K., & Morita, Y. (2023). Large language model-based system to provide immediate feedback to students in flipped classroom preparation learning. arXiv. https://doi.org/10.48550/arXiv.2307.11388
van Dinter, R., Tekinerdogan, B., & Catal, C. (2021). Automation of systematic literature reviews: A systematic literature review. Information and Software Technology, 136, Article 106589. https://doi.org/10.1016/j.infsof.2021.106589
Woo, J. H., & Choi, H. (2021). Systematic review for AI-based language learning tools. Journal of Digital Contents Society, 22(11), 1783–1792. https://doi.org/10.9728/dcs.2021.22.11.1783
Xiao, Y., & Watson, M. (2019). Guidance on conducting a systematic literature review. Journal of Planning Education and Research, 39(1), 93–112. https://doi.org/10.1177/0739456X17723971
Zhang, Y. (2021). Study on college English teaching reform in the age of artificial intelligence. In T. Volodina, R. Hou, & Y. Zhang (Eds.), Proceedings of the 1st International Conference on Education: Current Issues and Digital Technologies (ICECIDT 2021) (pp. 300–306). Atlantis Press. https://doi.org/10.2991/assehr.k.210527.052
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 authors
This work is licensed under a Creative Commons Attribution 4.0 International License.
