Comparative Performance of GPT-4, RAG-Augmented GPT-4, and Students in MOOCs [r-libre/3266]

Miladi, Fatma; Psyché, Valéry et Lemire, Daniel (sous presse). Comparative Performance of GPT-4, RAG-Augmented GPT-4, and Students in MOOCs. Dans Workshop on Breaking Barriers with Generative Intelligence (BBGI). Springer.

Fichier(s) associé(s) à ce document :

PDF - Comparative Performance of GPT-4.pdf
Contenu du fichier : Manuscrit soumis (avant évaluation)
Licence : Creative Commons CC BY.

Télécharger

Catégorie de document :	Communications dans des actes de congrès/colloques
Évaluation par un comité de lecture :	Oui
Étape de publication :	Accepté (sous presse)
Résumé :	Generative Pretrained Transformers (GPT) have significantly improved natural language processing, showcasing enormous versatility across diverse applications. Although GPT models have enormous po- tential, they frequently encounter issues such as mistakes and hallucina- tions, which may limit their practical use. Addressing these shortcomings, Retrieval-Augmented Generation (RAG) represents an innovative ap- proach that potentially enhances the accuracy and reliability of these models by leveraging external databases to correct and enrich their out- puts. In our study, a RAG-augmented GPT-4 model was tested within an AI-focused Massive Open Online Course (MOOC) and outperformed a standard GPT-4 model, achieving an 85% success rate compared to 81%. Notably, it also surpassed the average student performance, under- scoring its ability to deliver precise and contextually relevant responses. These findings suggest the potential of RAG in enhancing AI models for educational use and indicate that instructors can leverage this technol- ogy to refine assessment methods and that students can achieve more personalized and engaging learning experiences.
Déposant:	Lemire, Daniel
Responsable :	Daniel Lemire
Dépôt :	27 mai 2024 14:49
Dernière modification :	27 mai 2024 14:49

Actions (connexion requise)

RÉVISER