Evaluating Retrieval Quality in Retrieval-Augmented Generation
Alireza Salemi, Hamed Zamani
2024 · 118 citations
Evaluating retrieval-augmented generation (RAG) presents challenges, particularly for retrieval models within these systems. Traditional end-to-end evaluation methods are computationally expensive. Furthermore, evaluation of the retrieval model's performance based on query-document relevance labels shows a small correlation with the RAG system's downstream performance. We propose a novel evaluation approach, eRAG, where each document in the retrieval list is individually utilized by the large language model within the RAG system. The output generated for each document is then evaluated based…
Explore this paper's citation graph on Constellation.