Artificial intelligence-based large language models and integrity of exams and assignments in higher education: the case of tourism courses

• Ülkü, Abdullah ^[1]
  1. [1] Harran University, Turkey. School of Tourism and Hotel Management
• Localización: Tourism & Management Studies, ISSN-e 2182-8466, ISSN 2182-8458, Vol. 19, Nº. 4, 2023, págs. 21-34
• Idioma: inglés
• DOI: 10.18089/tms.2023.190402
• Títulos paralelos:
  • Inteligência artificial baseada em modelos de linguagem de grande escala e integridade de provas e trabalhos no ensino superior: o caso dos cursos de turismo
• Enlaces
  • Texto completo (pdf)
• Resumen
  • English

    There is an increasing concern regarding the potential misuse of ChatGPT-4 in compromising the integrity of examinations and assignments. This study aims to examine the capabilities of ChatGPT-4 in critical thinking abilities, whether it poses a threat to examinations and assignments in higher education, and create a discussion agenda on this issue. ChatGPT-4 was asked to generate, answer, and criticize questions in tourism marketing, tourism management, tourism economics, tourist guidance, and gastronomy. The answers were evaluated according to universal critical thinking standards. The findings obtained from this study showed that ChatGPT-4 had commendable competence in several critical thinking standards and could produce human-like texts.  However, there are certain domains that might be improved to comply more effectively with the expectations and norms of academia. Educators were recommended to use comprehensive approaches that combine technological and educational techniques to address the issue of cheating enabled by tools, such as ChatGPT-4, during assessment and exam processes.

  • português

    Há uma crescente preocupação em relação ao possível uso indevido do ChatGPT-4 comprometendo a integridade de exames e trabalhos académicos. Este estudo visa examinar as capacidades do ChatGPT-4 nas capacidades de pensamento crítico, avaliar se ele representa uma ameaça para exames e trabalhos no ensino superior e criar uma agenda de discussão sobre esse problema. Solicitou-se ao ChatGPT-4 que gerasse, respondesse e criticasse perguntas nas áreas de marketing turístico, gestão do turismo, economia do turismo, informação turística e gastronomia. As respostas foram avaliadas deacordo com padrões universais de pensamento crítico. Os resultados deste estudo mostraram que o ChatGPT-4 apresentou competência admirável em vários padrões de pensamento crítico e conseguiu produzir textos semelhantes aos humanos. No entanto, há certos domínios que podem ser aperfeiçoados para atender mais efetivamente às expetativas e normas académicas. Recomenda-se que os educadores adotem abordagens abrangentes que combinem técnicas tecnológicas e educacionais para lidar com o problema da fraude, facilitada por ferramentas como o ChatGPT-4, durante processos de avaliação e exames.

• Referencias bibliográficas
  • Ababneh, K. I., Ahmed, K., & Dedousis, E. (2022). Predictors of cheating in online exams among business students during the Covid pandemic:...
  • Adzima, K. (2021). Examining online cheating in higher education using traditional classroom cheating as a guide. Electronic Journal of E-Learning,...
  • Agasisti, T., & Soncin, M. (2021). Higher education in troubled times: On the impact of Covid-19 in Italy. Studies in Higher Education,...
  • AlAfnan, M. A., Dishari, S., Jovic, M. & Lomidze, K. (2023). Chatgpt as an educational tool: opportunities, challenges, and recommendations...
  • Ali, K., Barhom, N., Marino, F. T., & Duggal, M. (2023). The thrills and chills of chatgpt: ?mplications for assessments in undergraduate...
  • Atlas, S. (2023). Chatgpt for higher education and professional development: a guide to conversational AI. University of Rhode Island DigitalCommons@URI....
  • Barrientos, A., Cuadros, M., Alba, J., & Cruz, A. S. (2021, October 27-29). Implementation of a remote system for the supervision of online...
  • Barrot, J. S. (2023). Using ChatGPT for second language writing: Pitfalls and potentials. Assessing Writing, 57, 100745. https://doi.org/10.1016/j.asw.2023.100745
  • Benvenuti, M., Cangelosi, A., Weinberger, A., Mazzoni, E., Benassi, M., Barbaresi, M., & Orsoni, M. (2023). Artificial intelligence and...
  • Bernabei, M., Colabianchi, S., Falegnami, A., & Costantino, F. (2023). Students’ use of large language models in engineering education:...
  • Carvalho, L., Martinez-Maldonado, R., Tsai, Y.-S., Markauskaite, L., & De Laat, M. (2022). How can we design for learning in an AI world?...
  • Chirumamilla, A., & Sindre, G. (2019). Mitigation of cheating in online exams: Strengths and limitations (A. V. S. Kumar, Ed.). IGI Global....
  • Choi, J. H., Hickman, K. E., Monahan, A., & Schwarcz, D. B. (2023). ChatGPT goes to law school. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4335905
  • Coghlan, S., Miller, T., & Paterson, J. (2020). Good proctor or “Big Brother”? AI ethics and online exam supervision technologies. https://arxiv.org/abs/2011.07647.
  • Comas-Forgas, R., Lancaster, T., Calvo-Sastre, A., & Sureda-Negre, J. (2021). Exam cheating and academic integrity breaches during the...
  • Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations...
  • Crawford, J., Cowling, M., Central Queensland University, Australia, Allen, K.-A., & Monash University, Australia. (2023). Leadership...
  • Currie, G. M. (2023). Academic integrity and artificial intelligence: Is ChatGPT hype, hero or heresy? Seminars in Nuclear Medicine, 53(5),...
  • David, A., Mihai, D., Mihailescu, M.-E., Carabas, M., & Tapus, N. (2022). Scalability through distributed deployment for moodle learning...
  • Dendir, S., & Maxwell, R. S. (2020). Cheating in online courses: Evidence from online proctoring. Computers in Human Behavior Reports,...
  • Dias, J., Kamdi, D., Gharat, N., & Chudhari, P. (2019). Chatbot for Government Examination using AI. Journal of Engineering, 9(4), 58-62.
  • Duhaim, A. M., Al-mamory, S. O., & Mahdi, M. S. (2022). Cheating detection in online exams during Covid-19 pandemic using data mining...
  • Eke, D. O. (2023). ChatGPT and the rise of generative AI: Threat to academic integrity? Journal of Responsible Technology, 13, 100060. https://doi.org/10.1016/j.jrt.2023.100060
  • Elbanna, S., & Armstrong, L. (2023). Exploring the integration of ChatGPT in education: Adapting for the future. Management & Sustainability:...
  • Essel, H. B., Vlachopoulos, D., Tachie-Menson, A., Johnson, E. E., & Baah, P. K. (2022). The impact of a virtual teaching assistant (chatbot)...
  • Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2023). A SWOT analysis of ChatGPT: Implications for educational practice and...
  • Geerling, W., Mateer, G. D., Wooten, J., & Damodaran, N. (2023a). ChatGPT has Mastered the Principles of Economics: Now What? SSRN Electronic...
  • Gerashchenko, I. P., & Kovalev, V. A. (2021). Formation of educational ecosystems through the digital transformation of the educational...
  • Gilson, A., Safranek, C., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2022). How Does ChatGPT Perform on the Medical...
  • Gökta?, L. S. (2023). ChatGPT uzaktan e?itim s?navlar?nda ba?ar?l? olabilir mi turizm alan?nda do?ruluk ve do?rulama üzerine bir ara?t?rma...
  • Hashim, H. (2018). Application of technology in the digital era education. International Journal of Research in Counseling and Education,...
  • Hill, B. (2023). Taking the help or going alone: ChatGPT and class assignments. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4465833
  • Holden, O. L., Norris, M. E., & Kuhlmeier, V. A. (2021). Academic ?ntegrity in online assessment: A research review. Frontiers in Education,...
  • Hwang, A. (2018). Online and Hybrid Learning. Journal of Management Education, 42(4), 557–563. https://doi.org/10.1177/1052562918777550
  • Ilynykh, S. (2021). Online education: Trends and risks of digital technologies: VIII International Scientific and Practical Conference “Current...
  • Jungherr, A. (2023). Using ChatGPT and Other Large Language Model (LLM) applications for academic paper assignments [Preprint]. SocArXiv....
  • King, D. L., & Case, C. J. (2014). E-Cheating: Incidence and trends among college students. Issues In Information Systems, 15(1). https://doi.org/10.48009/1_iis_2014_20-27
  • Klijn, F., Mdaghri Alaoui, M., & Vorsatz, M. (2022). Academic integrity in on-line exams: Evidence from a randomized field experiment....
  • Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J.,...
  • Lanier, M. M. (2006). Academic integrity and distance learning. Journal of Criminal Justice Education, 17(2), 244–261. https://doi.org/10.1080/10511250600866166
  • Latifah, R., Budiyanto, C. W., & Saputro, H. (2022). Digital transformation readiness in education: A review. International Journal of...
  • Lin, J. C., Younessi, D. N., Kurapati, S. S., Tang, O. Y., & Scott, I. U. (2023). Comparison of GPT-3.5, GPT-4, and human user performance...
  • Maslov, I., Nikou, S., & Hansen, P. (2021). Exploring user experience of learning management system. The International Journal of Information...
  • Mellieon, Jr, H. I., & Robinson, P. A. (2021). The New norm: Faculty Perceptions of condensed online learning. American Journal of Distance...
  • Misra, D. P., & Chandwar, K. (2023). ChatGPT, artificial intelligence and scientific writing: What authors, peer reviewers and editors...
  • Montenegro-Rueda, M., Luque-de La Rosa, A., Sarasola Sánchez-Serrano, J. L., & Fernández-Cerero, J. (2021). Assessment in higher education...
  • Naidu, K., & Sevnarayan, K. (2023). ChatGPT: An ever-increasing encroachment of artificial intelligence in online assessment in distance...
  • Newton, P. M., & Xiromeriti, M. (2023). ChatGPT Performance on MCQ Exams in Higher Education. A Pragmatic Scoping Review [Preprint]. https://doi.org/10.35542/osf.io/sytu3
  • Nguyen, X.-A., Nguyen, S.-N., Nguyen, T.-T.-T., Luong, D.-H., & Tran, T.-G. (2022). Exploring Some academic dishonesty in remote online...
  • Nieuwoudt, J. E. (2020). Investigating synchronous and asynchronous class attendance as predictors of academic success in online education....
  • Noorbehbahani, F., Mohammadi, A., & Aminazadeh, M. (2022). A systematic review of research on cheating in online exams from 2010 to 2021....
  • Oravec, J. A. (2023). Artificial intelligence implications for academic cheating: expanding the dimensions of responsible human-AI collaboration...
  • Paul, R. (2005). The state of critical thinking today. New Directions for Community Colleges, 2005(130), 27–38. https://doi.org/10.1002/cc.193
  • Paul, R., & Elder, L. (2013). Critical thinking: Intellectual standards essential to reasoning well within every domain of human thought,...
  • Perera, P., & Lankathilaka, M. (2023). AI in higher education: A literature review of chatgpt and guidelines for responsible implementation....
  • Pérez, J. Q., Daradoumis, T., & Puig, J. M. M. (2020). Rediscovering the use of chatbots in education: A systematic literature review....
  • Petrolo, D., Fakhar Manesh, M., & Palumbo, R. (2023). Unpacking business, management, and entrepreneurship education online: Insights...
  • Putra, F. W., Rangka, I. B., Aminah, S., & Aditama, M. H. R. (2023). ChatGPT in the higher education environment: Perspectives from the...
  • Rahm, L. (2023). Educational imaginaries: Governance at the intersection of technology and education. Journal of Education Policy, 38(1),...
  • Skavronskaya, L., Hadinejad, A. (Hana), & Cotterell, D. (2023). Reversing the threat of artificial intelligence to opportunity: A discussion...
  • Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook Messenger. Computers &...
  • Susnjak, T. (2022). ChatGPT: The End of Online Exam Integrity?. http://arxiv.org/abs/2212.09292
  • Swiecki, Z., Khosravi, H., Chen, G., Martinez-Maldonado, R., Lodge, J. M., Milligan, S., Selwyn, N., & Gaševi?, D. (2022). Assessment...
  • Tsai, M.-L., Ong, C. W., & Chen, C.-L. (2023). Exploring the use of large language models (LLMs) in chemical engineering education: Building...
  • Ventayen, R. J. M. (2023). ChatGPT by OpenAI: Students’ viewpoint on cheating using Artificial Intelligence-Based application. SSRN Electronic...
  • Von Grunigen, D., De Azevedo E Souza, F. B., Pradarelli, B., Magid, A., & Cieliebak, M. (2018). Best practices in e-assessments with a...
  • Wagner, G., Lukyanenko, R., & Paré, G. (2022). Artificial intelligence and the conduct of literature reviews. Journal of Information Technology,...
  • Watson, G. R., & Sottile, J. (2010). Cheating in the digital age: do students cheat more in online courses? https://mds.marshall.edu/cgi/viewcontent.cgi?article=1000&context=eft_faculty
  • Wiley. (2020). White paper: Academic integrity in the age of online learning. https://www.wiley.com/en-us/network/education/instructors/teaching-strategies/academic-integrity-in-the-age-of-online-learning-3
  • Xie, X., Siau, K., & Nah, F. F.-H. (2020). COVID-19 pandemic – online education in the new normal and the next normal. Journal of Information...
  • Yau, C., & Chan, K. (2023, February 17). University of Hong Kong temporarily bans students from using ChatGPT, other AI-based tools for...
  • Yorkovsky, Y., & Levenberg, I. (2022). Distance learning in science and mathematics—Advantages and disadvantages based on pre-service...
  • Zaitsu, W., & Jin, M. (2023). Distinguishing ChatGPT(-3.5, -4)-generated and human-written papers through Japanese stylometric analysis....