Comparative Performance of GPT-4, RAG-Augmented GPT-4, and Students in MOOCs [r-libre/3266]

Miladi, Fatma; Psyché, Valéry, & Lemire, Daniel (2024). Comparative Performance of GPT-4, RAG-Augmented GPT-4, and Students in MOOCs. In Breaking Barriers with Generative Intelligence. Using GI to Improve Human Education and Well-Being (BBGI 2024). Springer, coll. « Communications in Computer and Information Science ». https://doi.org/10.1007/978-3-031-65996-6_7

File(s) available for this item:

PDF - Comparative Performance of GPT-4.pdf
Content : Submitted Version
License : Creative Commons Attribution.

Download

Item Type:	Papers in Conference Proceedings
Refereed:	Yes
Status:	Published
Abstract:	Generative Pretrained Transformers (GPT) have significantly improved natural language processing, showcasing enormous versatility across diverse applications. Although GPT models have enormous po- tential, they frequently encounter issues such as mistakes and hallucina- tions, which may limit their practical use. Addressing these shortcomings, Retrieval-Augmented Generation (RAG) represents an innovative ap- proach that potentially enhances the accuracy and reliability of these models by leveraging external databases to correct and enrich their out- puts. In our study, a RAG-augmented GPT-4 model was tested within an AI-focused Massive Open Online Course (MOOC) and outperformed a standard GPT-4 model, achieving an 85% success rate compared to 81%. Notably, it also surpassed the average student performance, under- scoring its ability to deliver precise and contextually relevant responses. These findings suggest the potential of RAG in enhancing AI models for educational use and indicate that instructors can leverage this technol- ogy to refine assessment methods and that students can achieve more personalized and engaging learning experiences.
Depositor:	Lemire, Daniel
Owner / Manager:	Daniel Lemire
Deposited:	27 May 2024 14:49
Last Modified:	12 Feb 2025 15:49

Actions (login required)

RÉVISER