ESL Pre-Service Teachers’ Literacy and Acceptance of ChatGPT as a Generative AI Tool for Writing: A Sequential Explanatory Study

Authors

  • Lea Me Balatero Davao de Oro State College, Philippines
  • Jessie Rose Jayno Davao de Oro State College, Philippines
  • Khif Muamar Miranda Davao de Oro State College, Philippines

DOI:

https://doi.org/10.54536/jtel.v2i3.3365

Keywords:

ChatGPT, ESL, Generative AI, Pre-Service Teachers, Writing

Abstract

The main objective of this research is to explore how ESL pre-service teachers perceived ChatGPT, a generative AI tool for writing, and the factors shaping their perception of ChatGPT for writing skills in an educational context. The research aims to investigate ESL pre-service teachers’ awareness, acceptance, perception, and the interplay between these factors regarding ChatGPT for writing. A sequential explanatory research design was employed and utilized complete enumeration sampling. The sample consisted of 80 undergraduate students enrolled in the Teacher Education Department English majors were used to collect students’ engagement data including students’ perceptions and usage of the platform, and user experiences were examined using the data using a 5-point Likert scale. Also, an in-depth interview was conducted with ten (10) students based on those who got the highest and lowest scorers in the Quantitative phase to understand deeper experiences in using ChatGPT and to explore design opportunities for leveraging ChatGPT as a generative AI tool for writing in the field of ESL education. The findings emphasized the awareness and acceptance of ChatGPT in writing. However, there were limitations to consider. These challenges encompassed issues such as inaccurate automated responses and the possibility of plagiarism. Despite these limitations, the research findings demonstrate that ChatGPT is effective writing tool. As researchers who experienced the process of different writing styles using ChatGPT, researchers believe that its potential needs to be maximally utilized. We suggest its application across different subjects and disciplines to examine its strengths and weaknesses in depth thoroughly.

References

AlAfnan, M. A., Dishari, N. S., Jovic, N. M., & Lomidze, N. K. (2023). ChatGPT as an educational tool: Opportunities, challenges, and recommendations for communication, business writing, and composition courses. Journal of Artificial Intelligence and Technology. https://doi.org/10.37965/jait.2023.0184

AlAfnan, M. A., & MohdZuki, S. F. (2023). Do Artificial Intelligence Chatbots have a writing style? an investigation into the stylistic features of ChatGPT-4. Journal of Artificial Intelligence and Technology. https://doi.org/10.37965/jait.2023.0267

Alawida, M., Shawar, B. A., Abiodun, O. I., Mehmood, A., Omolara, A. E., & Hwaitat, A. K. A. (2023). Unveiling the dark side of ChatGPT: Exploring cyberattacks and enhancing user awareness. Information, 15(1), 27. https://doi.org/10.3390/info15010027

Albayati, H. (2024). Investigating undergraduate students’ perceptions and awareness of using ChatGPT as a regular assistance tool: A user acceptance perspective study. Computers and Education. Artificial Intelligence, 6, 100203. https://doi.org/10.1016/j.caeai.2024.100203

Algaraady, J., & Mahyoob, M. (2023). ChatGPT’s capabilities in spotting and analyzing writing errors experienced by EFL Learners. Arab World English Journal, 9, 3–17. https://doi.org/10.24093/awej/call9.1

Alipio, M., Lantajo, G. M., Pregoner, J. D. (2023). On the use of ChatGPT in health science education: opportunities and obstacles. IMCC Journal of Science, 3(2), 1-7. SSRN: https://ssrn.com/abstract=4588412

Aljabr, Fahad (2023). Gauging the Saudi EFL Learners Level of Awareness and Attitudes towards the Use of ChatGPT. (2023c). Information Sciences Letters /Information Sciences Letters, 12(11), 3101–3109. https://doi.org/10.18576/isl/121120

Almahdawi, A. J. (2024). The Role of Artificial Intelligence in Developing Digital Transformation Skills Language Communication, and Scientific Trends among Students of the College of Education at Al Ain University. American Journal of Education and Technology, 3(1), 25–34. https://doi.org/10.54536/ajet.v3i1.2187

Almeida, F. (2018). Strategies to perform a mixed methods study. Zenodo. https://doi.org/10.5281/zenodo.1406214

Akbarani, R., Pamungkas, F., D. (2023). The Challenges of Using Artificial Intelligence in Academic Writing. Prosiding Annual International Conference on Islamic Education (AICIED), 01(64-72). https://prosiding.iainponorogo.ac.id/index.php/aicied/article/view/1051

Azaria, A., Azoulay, R., & Reches, S. (2023). ChatGPT is remarkable Tool – For Experts. arXiv. https://doi.org/10.48550/arxiv.2306.03102

Baskara, F. R. (2023). Integrating ChatGPT into EFL Writing Instruction: Benefits and Challenges. International Journal of Education and Learning, 5(1), 44–55. https://doi.org/10.31763/ijele.v5i1.858

Baskara, R. & Mukarto, M. (2023). Exploring the Implications of ChatGPT for Language Learning in Higher Education. Indonesian Journal of English Language Teaching and Applied Linguistics, 7(2), 343-358. http://dx.doi.org/10.21093/ijeltal.v7I2.1387

Bhatia, P. (2023). ChatGPT for Academic Writing: A Game Changer or a Disruptive Tool? Journal of Anaesthesiology Clinical Pharmacology, 39(1), 1. https://doi.org/10.4103/joacp.joacp_84_23

Bonsu, E. M., & Baffour-Koduah, D. (2023). From the consumers’ side: Determining students’ perception and intention to use ChatGPT in Ghanaian Higher Education. Journal of Education Society & Multiculturalism, 4(1), 1–29. https://doi.org/10.2478/jesm-2023-0001

Bouzar, A., Idrissi, K. E., & Ghourdou, T. (2024). Investigating the Correlation between Different ChatGPT Versions and Task Initiation among Postgraduate Students: A Cross-Sectional Study. American Journal of Education and Technology, 3(2), 79–84. https://doi.org/10.54536/ajet.v3i2.2704

Caratiquit, K. D., & Caratiquit, L. J. C. (2023). ChatGPT as an academic support tool on the academic performance among students: The mediating role of learning motivation. Journal of Social, Humanity, and Education, 4(1), 21–33. https://doi.org/10.35912/jshe.v4i1.1558

Carolus, A., Koch, M., Straka, S., Latoschik, M. E., & Wienrich, C. (2023). MAILS – Meta AI Literacy Scale: Development and testing of an AI Literacy questionnaire based on Well-Founded Competency Models and Psychological Change- and Meta-Competencies. arXiv. https://doi.org/10.48550/arxiv.2302.09319

Casal-Otero, L., Catala, A., Fernández-Morante, C. et al., (2023). AI literacy in K-12: A Systematic Literature Review. IJ STEM Ed 10, 29. https://doi.org/10.1186/s40594-023-00418-7

Chen, M., & Li, G. (2023). ChatGPT for Mechanobiology and Medicine: A Perspective. Mechanobiology in Medicine, 1(1), 100005. https://doi.org/10.1016/j.mbm.2023.100005

Christensen, M. H., Kristensen, L. B., Vinther, N. M., & Boye, K. (2021). Grammar is background in sentence processing. Language and Cognition, 13(1), 128–153. https://doi:10.1017/langcog.2020.30

Creswell, J. W., & Plano Clark, V. L. (2018). Designing and Conducting Mixed Methods Research (3rd ed.). Sage Publication https://files.eric.ed.gov/fulltext/ED611786.pdf

Dahri, N. A., Yahaya, N., Al-Rahmi, W. M., Aldraiweesh, A., Alturki, U., Almutairy, S., Shutaleva, A., & Soomro, R. B. (2024). Extended TAM based acceptance of AI-Powered ChatGPT for supporting metacognitive self-regulated learning in Education: A Mixed-Methods Study. Heliyon, 10(8), e29317. https://doi.org/10.1016/j.heliyon.2024.e29317

Fabella, F. (2023). Attitudes toward the positive and negative features of ChatGPT by selected First Year College students. International Research Journal of Modernization in Engineering Technology and Science, 05(10) 2582-5208. https://doi.org/10.56726/IRJMETS45426

Garg, R. K., Urs, V. L., Agrawal, A. A., Chaudhary, S. K., Paliwal, V., & Kar, S. K. (2023). Exploring the role of ChatGPT in patient care (Diagnosis and Treatment) and Medical Research: A Systematic Review. medRxiv. https://doi.org/10.1101/2023.06.13.23291311

George, B., & Wooden, O. (2023). Managing the strategic transformation of higher education through Artificial Intelligence. Administrative Sciences, 13(9), 196. https://doi.org/10.3390/admsci13090196

Guo, Y., & Lee, D. (2023). Leveraging ChatGPT for enhancing critical thinking skills. Journal of Chemical Education, 100(12), 4876–4883. https://doi.org/10.1021/acs.jchemed.3c00505

Haleem, A., Javaid, M., & Singh, R. P. (2022). An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges. Bench Council Transactions on Benchmarks, Standards and Evaluations, 2(4), 100089. https://doi.org/10.1016/j.tbench.2023.100089

Hartmann, J., Schwenzow, J., & Witte, M. (2023). The political ideology of conversational AI: Converging evidence on ChatGPT’s pro-environmental, Left-Libertarian Orientation. arXiv. https://doi.org/10.48550/arXiv.2301.01768

Hasanein, A. M., & Sobaih, A. E. E. (2023). Drivers and consequences of CHATGPT use in Higher Education: Key Stakeholder Perspectives. European Journal of Investigation in Health, Psychology and Education, 13(11), 2599–2614. https://doi.org/10.3390/ejihpe13110181

Haseski, H.I. (2019). What do Turkish pre-service teachers think about Artificial Intelligence? International Journal of Computer Science Education in Schools, 3(2) ISSN 2513-8359. https://doi.org/10.21585/ijcses.v3i2.55

Herbold, S., Hautli-Janisz, A., Heuer, U., Kikteva, Z., & Trautsch, A. (2023). AI, write an essay for me: A large-scale comparison of human-written versus ChatGPT-generated essays. arXiv. https://doi.org/10.48550/arxiv.2304.14276

Hidayatullah, E. (2024). Evaluating the effectiveness of ChatGPT to improve English students’ writing skills. Humanities, Education, Applied Linguistics, and Language Teaching: Conference Series, 1(1), 2024, pp. 14-19. https://seminar.ustjogja.ac.id/index.php/heal

Hua, S., Jin, S., & Jiang, S. (2023). The limitations and ethical considerations of CHATGPT. Data Intelligence, 1–38. https://doi.org/10.1162/dint_a_00243

Iqbal, N., Ahmed, H., & Azhar, K. A. (2022). Exploring Teachers’ Attitudes Towards Using ChatGPT. Global Journal for Management and Administrative Sciences, 3(4), 97–111. https://doi.org/10.46568/gjmas.v3i4.163

Kaliyadan, F., & Seetharam, K. (2023). ChatGPT- Quo vadis? Indian Dermatology Online Journal, 14(4), 457. https://doi.org/10.4103/idoj.idoj_344_23

Kasinidou, M. (2023). AI literacy for all: A participatory approach. In Proceedings of the Conference on Innovation and Technology in Computer Science Education (Vol. 2, pp. 607-608). https://doi.org/10.1145/3587103.3594135

Kleebayoon, A., & Wiwanitkit, V. (2023). Chatbots in Medical Research. Clinical Nuclear Medicine. https://doi.org/10.1097/rlu.0000000000004877

Kleesiek, J., Wu, Y., Štiglic, G., Egger, J., & Bian, J. (2023). An Opinion on ChatGPT in health care—written by humans only. The Journal of Nuclear Medicine, 64(5), 701–703. https://doi.org/10.2967/jnumed.123.265687

Kong, SC., Cheung, W.M., Zhang, G. (2022). Evaluating Artificial Intelligence literacy courses for fostering conceptual learning, literacy and empowerment in university students: Refocusing to conceptual building. Computers in Human Behavior Reports. https://doi.org/10.1016/j.chbr.2022.100223

Koubaa, A., Boulila, W., Ghouti, L., Alzahem, A., & Latif, S. (2023). Exploring ChatGPT capabilities and limitations: A survey. IEEE Access, 11, 118698–118721. https://doi.org/10.1109/access.2023.3326474

Kothgassner, O. D., & Felnhofer, A. (2023). ChatGPT, Who? Digital Psychology, 4(1), 1. https://doi.org/10.24989/dp.v4i1.2268

Laupichler, M. C., Aster, A., Schirch, J., & Raupach, T. (2022). Artificial intelligence literacy in higher and adult education: A scoping literature review. Computers & Education: Artificial Intelligence, 3, 100101. https://doi.org/10.1016/j.caeai.2022.100101

Leunard, H., Rachmawati, R., Zani, B. N., & Maharjan, K. (2023). GPT Chat: Opportunities and challenges in the learning process of arabic language in higher education. Journal International of Lingua and Technology, 2(1), 10–22. https://doi.org/10.55849/jiltech.v2i1.439

Lingard, L. (2023). Writing with ChatGPT: An illustration of its capacity, limitations & implications for academic writers. Perspectives on Medical Education, 12(1), 261–270. https://doi.org/10.5334/pme.1072

Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), 410. https://doi.org/10.3390/educsci13040410

Long, D., & Magerko, B. (2020). What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-14). ACM. https://doi.org/10.1145/3313831.3376727

Lund, B., & Wang, T. (2023). Chatting about ChatGPT: How may AI and GPT impact academia and libraries? Social Science Research Network. https://doi.org/10.2139/ssrn.4333415

Maboloc, C. R. (2023). Chat GPT: The Need for an Ethical Framework to Regulate its Use in Education. Journal of Public Health. https://doi.org/10.1093/pubmed/fdad125

Menon, D., & Shilpa, K. (2023). “Chatting with ChatGPT”: Analyzing the factors influencing users’ intention to use the open AI’s ChatGPT using the UTAUT Model. Heliyon, 9(11), e20962. https://doi.org/10.1016/j.heliyon.2023.e20962

Meyer, J. G., Urbanowicz, R. J., Martin, P., O’Connor, K., Li, R., Peng, P., . . . Moore, J. H. (2023). ChatGPT and Large Language Models in Academia: Opportunities and Challenges. BioData. https://doi.org/10.1186/s13040-023-00339-9

Michaelides, D. T. (1997). Exact tests via complete enumeration: A distributed computing approach (Doctoral dissertation, University of Southampton). https://eprints.soton.ac.uk/250749/

Mondal, H., & Mondal, S. (2023b). ChatGPT in Academic Writing: Maximizing its Benefits and Minimizing the Risks. Indian Journal of Ophthalmology, 71(12), 3600–3606. https://doi.org/10.4103/ijo.ijo_718_23

Ng, D. T. K., Wenjie, W., Chiu, T. K. F., Leung, J. K L., Chu, S. K. W. (2023). Design and Validation of the AI Literacy Questionnaire: The affective, behavioural, cognitive, and ethical approach. British Journal of Educational Technology. https://doi.org/10.1111/bjet.13411

Obenza, B. N., Caballo, J. H. S., Caangay, R. B. R., Makigod, T. E. C., Almocera, S. M., Bayno, J. L. M., Camposano, J. J. R., Cena, S. J. G., Garcia, J. A. K., Labajo, B. F. M., & Tua, A. G. (2024). Analyzing university students’ attitude and behavior toward AI using the extended unified theory of acceptance and use of technology model. American Journal of Applied Statistics and Economics, 3(1), 99–108. https://doi.org/10.54536/ajase.v3i1.2510

Ofosu-Ampong, K., Acheampong, B., Kevor, M.-O., & Amankwah-Sarfo, F. (2023). Acceptance of Artificial Intelligence (ChatGPT) in Education: Trust, Innovativeness and Psychological Need of Students. (2023). Information and Knowledge Management. https://doi.org/10.7176/ikm/13-4-03

Oğurlu, Ü., & Mossholder, J. (2023). The Perception of ChatGPT among Educators: Preliminary Findings. Research in Social Sciences and Technology, 8(4), 196–215. https://doi.org/10.46303/ressat.2023.39

Osborne, N., & Grant-Smith, D. (2021). In-depth interviewing. In Cities research series (pp. 105–125). https://doi.org/10.1007/978-981-16-1677-8_7

Oviedo-Trespalacios, O., Peden, A. E., Cole-Hunter, T., Costantini, A., Haghani, M., Rod, J.... Reniers, G. (2023). The Risks of Using ChatGPT to Obtain Common Safety-Related Information and Advice. Safety Science, 167, 106244. https://doi.org/10.1016/j.ssci.2023.106244

Quintans‐Júnior, L. J., Gurgel, R. Q., De Souza Araújo, A. A., Filho, D. C., & Martins‐Filho, P. R. (2023). ChatGPT: The New Panacea of the Academic World. Revista Da Sociedade Brasileira De Medicina Tropical, 56. https://doi.org/10.1590/0037-8682-0060-2023

Rezaei, M., Salehi, H., & Tabatabaei, O. (2024). ChatGPT: A helpful scaffold or a debilitating crutch for academic writing? In Proceedings of the 11th International and 17th National Conference on E-Learning and E-Teaching (ICeLeT) (pp. 1-5). IEEE. https://doi.org/10.1109/ICeLeT62507.2024.10493087

Sa’adah, A. R. (2020). Writing skill in teaching english: An Overview. Educasia, 5(1), 21–35. https://doi.org/10.21462/educasia.v5i1.41

Sallam, M., Salim, N. A., Barakat, M., Al-Mahzoum, K., Al-Tammemi, A. B., Malaeb, D., Hallit, R., & Hallit, S. (2023). Assessing health students’ attitudes and usage of CHATGPT in Jordan: Validation study. JMIR Medical Education, 9, e48254. https://doi.org/10.2196/48254

Sallam, M. (2023). ChatGPT utility in healthcare education, research, and practice: Systematic review on the promising perspectives and valid concerns. Healthcare, 11, 887, https://doi:10.3390/healthcare11060887.

Sallam, M. (2023). The Utility of ChatGPT as an example of large language models in healthcare education, research and practice: Systematic review on the future perspectives and potential limitations. medRxiv.https://doi.org/10.1101/2023.02.19.23286155

Santiago, C. S., Embang, S. I., Acanto, R. B., Ambojia, K. W. P., Aperocho, M. D. B., Balilo, B. B., Cahapin, E. L., Conlu, M. T. N., Lausa, S. M., Laput, E. Y., Malabag, B. A., Paderes, J. J., & Romasanta, J. K. N. (2023). Utilization of writing assistance tools in research in selected higher learning institutions in the Philippines: A text Mining Analysis. International Journal of Learning, Teaching and Educational Research, 22(11), 259–284. https://doi.org/10.26803/ijlter.22.11.14

Schmidt-Fajlik, R. (2023). ChatGPT as a Grammar Checker for Japanese English Language Learners: A Comparison with Grammarly and Pro Writing Aid. AsiaCALL Online Journal, 14(1), 105–119. https://doi.org/10.54855/acoj.231417

Shahriar, S., & Hayawi, K. (2023). Let’s have a chat! A Conversation with ChatGPT: Technology, Applications, and Limitations. arXiv. https://doi.org/10.48550/arxiv.2302.13817

Shidiq, M. (2023). The Use of Artificial Intelligence-Based ChatGPT and its challenges for the world of education; From the viewpoint of the development of creative writing skills. Proceeding of International Conference on Education, Society and Humanity, 01(2986–5832). Retrieved from https://ejournal.unuja.ac.id/index.php/icesh /article/view/5614

Shoufan, A. (2023). Exploring Students’ Perceptions of CHATGPT: Thematic Analysis and Follow-up survey. IEEE Access, 11, 38805–38818. https://doi.org/10.1109/access.2023.3268224

Soelistiyowati, E., Permatasari, A. N., Nugroho, V. T. A., & Kusmulyadi, A. S. F. (2024). Enhancing college learners’ writing skills through the integration of ChatGPT: Strategies, benefits, and concerns. eralingua: Jurnal Pendidikan Bahasa Asing Dan Sastra, 8(1), 1. https://doi.org/10.26858/eralingua.v8i1.53461

Stepanechko, O., & Kozub, L. (2023). English teachers’ concerns about the ethical use of ChatGPT by University Students. Grail of Science, 25, 297–302. https://doi.org/10.36074/grail-of-science.17.03.2023.051

Strzelecki, A. (2023). Students’ Acceptance of CHATGPT in Higher Education: An Extended Unified Theory of Acceptance and Use of Technology. Innovative Higher Education. https://doi.org/10.1007/s10755-023-09686-1

Tran, T. N., & Tran, H. P. (2023). Exploring the role of chatgpt in developing critical digital literacies in language learning: A Qualitative study. Proceedings of the Asia CALL International Conference, 4, 1–17. https://doi.org/10.54855/paic.2341

Tseng, W., & Warschauer, M. (2023). AI-writing tools in education: if you can’t beat them, join them. Journal of China Computer-assisted Language Learning, 3(2), 258–262. https://doi.org/10.1515/jccall-2023-0008

Urban, M., Děchtěrenko, F., Lukavský, J., Hrabalová, V., Svacha, F., Brom, C., & Urban, K. (2024). ChatGPT improves creative problem-solving performance in university students: An experimental study. Computers and Education/Computers & Education, 215, 105031. https://doi.org/10.1016/j.compedu.2024.105031

Van Wyk, M. M., Adarkwah, M., & Amponsah, S. (2023). Why all the hype about ChatGPT? Academics’ views of a chat based conversational learning strategy at an open distance e-learning institution. Open Praxis, 15, 214–225. https://doi.org/10.55982/openpraxis.15.3.563

Yutong, N. Z., Leqi, N. Z., Rongxiao, N. Z., & Hang, N. S. (2024). Exploring AI-Generated feedback on english writing: A case study of ChatGPT. US-China Foreign Language, 22(3). https://doi.org/10.17265/1539-8080/2024.03.002

Zhai, X., Chu, X., Chai, C., Jong, M., Istenič, A., Spector, M., Liu, J., Yuan, J., & Li, Y. (2021). A review of artificial intelligence (AI) in education from 2010 to 2020. Complexity, 2021, 8812542. https://doi.org/10.1155/2021/8812542

Zhao, T., Chance, R., Buckhalter, C., & Wang, G. (2024). Impact of CHATGPT on student writing in construction management: Analyzing literature and countermeasures for writing intensive courses. EPiC Series in Built Environment. https://doi.org/10.29007/4d7m

Published

2024-09-21

How to Cite

Balatero, L. M., Jayno, J. R., & Miranda, K. M. (2024). ESL Pre-Service Teachers’ Literacy and Acceptance of ChatGPT as a Generative AI Tool for Writing: A Sequential Explanatory Study. Journal of Tertiary Education and Learning, 2(3), 13–29. https://doi.org/10.54536/jtel.v2i3.3365