ChatGPT displays a remarkable ability to match or even exceed the academic performance in nine out of 32 courses subjects, according to NYU Abu Dhabi researchers.
In the study, which was published in Scientific Reports, the researchers examined ChatGPT's performance against student responses across eight disciplines.
Led by NYUAD Associate Professor of Computer Science Talal Rahwan and Assistant Professor of Computer Science Yasir Zaki, 33 faculty members provided student-written answers for ten assessment questions for 32 different courses in those disciplines.
ChatGPT was then tasked to formulate responses for the same set of questions and those responses were evaluated by three impartial graders alongside student submissions, with the graders unaware of the answers' sources.
The results showed that ChatGPT's responses achieved equivalent or higher average grades compared to students in 9 out of the 32 courses.
Titled Perception, performance, and detectability of conversational artificial intelligence across 32 university courses, the study's findings shed light on the evolving landscape of education and technology, as well as the perceptions of both students and educators.
We found that generative AI tools such as ChatGPT pose a serious threat to the integrity of student evaluation processes. Currently, ChatGPT’s performance is comparable, if not superior, to that of students in multiple courses, suggesting that students may use ChatGPT to cheat in homework and exams.
Moreover, current AI-text classifiers cannot reliably detect ChatGPT’s use in school work, suggesting that there is no way to identify the students who use ChatGPT to cheat, according to the assistant professor of computer science.
“There seems to be an emerging consensus among students to use the tool, and among educators to treat its use as plagiarism. Our findings offer insights that could guide policy discussions addressing the integration of artificial intelligence into educational frameworks,” he said.
Conversely, educators across all surveyed nations appeared to underestimate students' inclination to leverage ChatGPT, with 70 percent of educators viewing its usage as potentially constituting plagiarism.
The study unveiled the misclassification rates of AI-generated text identification tools. Both GPTZero and Open-AI's text classifier mistakenly attributed ChatGPT-generated answers to human authors in 32 percent and 49 percent of cases, respectively.
This collective body of findings sheds light on the dynamic intersection of AI and education, offering insights that could potentially guide future policies in academic settings.
It is worth noting that mathematics and economics were the only disciplines where students consistently outperformed ChatGPT. However, the AI-powered tool demonstrated its superiority in the "Introduction to Public Policy" course, boasting an average grade of 9.56, dwarfing the students' 4.39 average.
The study also encompassed the perspectives of 1,601 individuals, featuring a diverse cohort of about 200 students and 100 educators from Brazil, India, Japan, the US, and the UK . A remarkable 74 percent of students expressed their willingness to utilize ChatGPT to assist with their homework assignments.
The contributing NYUAD faculty and researchers include:
- Research Assistant Hazem Ibrahim
- Graduate Research Assistant Fengyuan Liu
- Graduate Research Assistant Rohail Asim
- Post-Doctoral Associate Balaraju Battu
- Research Associate Sidahmed Benabderrahmane
- Graduate Research Assistant Bashar Alhafni
- Assistant Professor of Economics Wifag Adnan
- Assistant Professor of Computer Engineering Tuka Alhanai
- Assistant Professor of Computational Social Science Bedoor AlShebli
- Assistant Professor of Computer Science Riyadh Baghdadi
- Assistant Professor of Psychology Jocelyn J. Bélanger
- Clinical Professor of Mathematics Elena Beretta
- Assistant Professor of Civil and Urban Engineering Kemal Celik
- Lecturer of Computer Science Moumena Chaqfeh
- Professor of Mechanical Engineering Mohammed Daqaq
- Visiting Assistant Professor of Social Research and Public Policy Zaynab El Bernoussi
- Assistant Professor of Psychology Daryl Fougnie
- Assistant Professor of Civil and Urban Engineering Borja Garcia de Soto
- Professor of Practice in Mathematics Alberto Gandolf
- Assistant Professor of Electrical and Computer Engineering Andras Gyorgy
- Professor of Computer Science Nizar Habash
- Assistant Professor of Political Science Jonathan Andrew Harris
- Assistant Professor of Political Science Aaron Kaufman
- Visiting Professor of Computer Science Lefteris Kirousis
- Assistant Professor of Political Science Korhan Kocak
- Assistant Professor of Social Research and Public Policy Kangsan Lee
- Visiting Senior Lecturer of Social Research and Public Policy Seungah Sarah Lee
- Associate Professor of Economics Samreen Malik
- Associate Professor of Electrical and Computer Engineering Michail Maniatakos
- Professor of Psychology David Melcher
- Visiting Professor of Computer Science Azzam Mourad
- Assistant Professor of Social Research and Public Policy Minsu Park
- Associate Professor of Electrical and Computer Engineering Mahmoud Rasras
- Adjunct Assistant Professor Alicja Reuben
- Lecturer of Mathematics Dania Zantout
- Associate Professor of Practice of Political Science Nancy W. Gleason
- Assistant Professor of Social Research and Public Policy Kinga Makovi
- Associate Professor of Computer Science Talal Rahwan
- Assistant Professor of Computer Science Yasir Zaki
