Slide
Centre Interdisciplinaire
de Recherche et d’Innovation
en Cybersécurité et Société
de Recherche et d’Innovation
en Cybersécurité et Société
1.
Sadfi, Y.; Amamou, H.; Davoust, A.; Avila, A. R.
Comparative Analysis of Machine Learning, LLMs, and RAG for Fake News Detection Article d'actes
Dans: IEEE Conf. Artif. Intell., CAI, p. 2128–2133, Institute of Electrical and Electronics Engineers Inc., 2026, ISBN: 979-833156039-3 (ISBN), (Journal Abbreviation: IEEE Conf. Artif. Intell., CAI).
Résumé | Liens | BibTeX | Étiquettes: Classification (of information), Comparative analyzes, Comprehensive assessment, decision making, Decision-making process, Fake detection, Generalisation, High-accuracy, Language model, Learning algorithms, Learning systems, Machine learning, Machine learning algorithms, Machine-learning, Network platforms, Performance
@inproceedings{sadfiComparativeAnalysisMachine2026,
title = {Comparative Analysis of Machine Learning, LLMs, and RAG for Fake News Detection},
author = {Y. Sadfi and H. Amamou and A. Davoust and A. R. Avila},
url = {https://www.scopus.com/pages/publications/105042025800?origin=resultslist},
doi = {10.1109/CAI68641.2026.11536343},
isbn = {979-833156039-3 (ISBN)},
year = {2026},
date = {2026-01-01},
booktitle = {IEEE Conf. Artif. Intell., CAI},
pages = {2128–2133},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {The rapid spread of misinformation on social network platforms poses a serious threat to the decision-making processes in democratic societies. To mitigate the problem and cope with new misinformation scenarios several datasets have been proposed, followed by the emergence of new methods for detecting fake news. While several comparative analyses have been proposed to benchmark these solutions, the ever-growing development of new methods requires up-to-date analysis. For instance, recent approaches based on retrieval-augmented generation (RAG) have yet to be assessed and compared to previous methods used for fake news detection. In this study, we aim to fill this gap by presenting a comprehensive assessment of classical machine learning (ML) algorithms, Transformer-based models, such as BERT, large language models (LLMs), and RAG-based classifiers. We adopt four traditional fake news datasets, including a Portuguese one. Results indicate that classical ML and BERT deliver strong performance on in-domain tasks, demonstrating efficiency and high accuracy. RAG-augmented LLMs, on the other hand, offer enhanced generalization and robustness in cross-domain and cross-dataset settings, but are outperformed by traditional approaches for in-domain evaluation. © 2026 IEEE.},
note = {Journal Abbreviation: IEEE Conf. Artif. Intell., CAI},
keywords = {Classification (of information), Comparative analyzes, Comprehensive assessment, decision making, Decision-making process, Fake detection, Generalisation, High-accuracy, Language model, Learning algorithms, Learning systems, Machine learning, Machine learning algorithms, Machine-learning, Network platforms, Performance},
pubstate = {published},
tppubtype = {inproceedings}
}
The rapid spread of misinformation on social network platforms poses a serious threat to the decision-making processes in democratic societies. To mitigate the problem and cope with new misinformation scenarios several datasets have been proposed, followed by the emergence of new methods for detecting fake news. While several comparative analyses have been proposed to benchmark these solutions, the ever-growing development of new methods requires up-to-date analysis. For instance, recent approaches based on retrieval-augmented generation (RAG) have yet to be assessed and compared to previous methods used for fake news detection. In this study, we aim to fill this gap by presenting a comprehensive assessment of classical machine learning (ML) algorithms, Transformer-based models, such as BERT, large language models (LLMs), and RAG-based classifiers. We adopt four traditional fake news datasets, including a Portuguese one. Results indicate that classical ML and BERT deliver strong performance on in-domain tasks, demonstrating efficiency and high accuracy. RAG-augmented LLMs, on the other hand, offer enhanced generalization and robustness in cross-domain and cross-dataset settings, but are outperformed by traditional approaches for in-domain evaluation. © 2026 IEEE.
2.
Amamou, H.; Gagnon, S.; Davoust, A.; Avila, A. R.
Towards Robust Retrieval-Augmented Generation Based on Knowledge Graph: A Comparative Analysis Article d'actes
Dans: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern., p. 80–85, Institute of Electrical and Electronics Engineers Inc., 2025, ISBN: 1062922X (ISSN); 979-833153358-8 (ISBN), (Journal Abbreviation: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.).
Résumé | Liens | BibTeX | Étiquettes: Benchmarking, Comparative analyzes, External sources, Generation systems, Information integration, Information retrieval, Knowledge graph, Knowledge graphs, Language model, Model response, Noise robustness, Pre-training, Prior-knowledge
@inproceedings{amamouRobustRetrievalAugmentedGeneration2025,
title = {Towards Robust Retrieval-Augmented Generation Based on Knowledge Graph: A Comparative Analysis},
author = {H. Amamou and S. Gagnon and A. Davoust and A. R. Avila},
url = {https://www.scopus.com/pages/publications/105033145604?origin=resultslist},
doi = {10.1109/SMC58881.2025.11343466},
isbn = {1062922X (ISSN); 979-833153358-8 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.},
pages = {80–85},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {Retrieval-Augmented Generation (RAG) was first introduced to enhance the capabilities of Large Language Models (LLMs) beyond their encoded-prior knowledge. This is achieved by providing LLMs with an external source of knowledge, which helps to reduce factual hallucinations and enables the access to new information, typically not available during their pretraining phase. Despite its benefits, there is an increasing concern with the impact of inconsistent retrieved information towards LLMs' responses. Hence, the Retrieval-Augmented Generation Benchmark (RGB) was introduced as a new testbed for RAG evaluation, meant to assess the robustness of LLMs towards inconsistency in the retrieved information. In this work, we use the RGB corpus to evaluate LLMs in four scenarios: (1) noise robustness; (2) information integration; (3) negative rejection; and (4) counterfactual robustness. We perform a comparative analysis between the RAG baseline defined by the RGB and variations of GraphRAG, which is a RAG system based on a Knowledge Graph (KG) and developed to retrieve relevant information from large documents. We tested GraphRAG under three customization to improve its robustness. Our approach demonstrates improvements compared to the RGB baseline, providing insights on how to design more reliable RAG systems, tailored for real-world scenarios. © 2025 IEEE.},
note = {Journal Abbreviation: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.},
keywords = {Benchmarking, Comparative analyzes, External sources, Generation systems, Information integration, Information retrieval, Knowledge graph, Knowledge graphs, Language model, Model response, Noise robustness, Pre-training, Prior-knowledge},
pubstate = {published},
tppubtype = {inproceedings}
}
Retrieval-Augmented Generation (RAG) was first introduced to enhance the capabilities of Large Language Models (LLMs) beyond their encoded-prior knowledge. This is achieved by providing LLMs with an external source of knowledge, which helps to reduce factual hallucinations and enables the access to new information, typically not available during their pretraining phase. Despite its benefits, there is an increasing concern with the impact of inconsistent retrieved information towards LLMs' responses. Hence, the Retrieval-Augmented Generation Benchmark (RGB) was introduced as a new testbed for RAG evaluation, meant to assess the robustness of LLMs towards inconsistency in the retrieved information. In this work, we use the RGB corpus to evaluate LLMs in four scenarios: (1) noise robustness; (2) information integration; (3) negative rejection; and (4) counterfactual robustness. We perform a comparative analysis between the RAG baseline defined by the RGB and variations of GraphRAG, which is a RAG system based on a Knowledge Graph (KG) and developed to retrieve relevant information from large documents. We tested GraphRAG under three customization to improve its robustness. Our approach demonstrates improvements compared to the RGB baseline, providing insights on how to design more reliable RAG systems, tailored for real-world scenarios. © 2025 IEEE.