Bauhaus Universitat Weimar Papers

Browse 1 peer-reviewed paper from Bauhaus Universitat Weimar spanning Crowdsourcing, RAG Evaluation (2025). Research powered by Prolific's high-quality participant data.

This page lists 1 peer-reviewed paper from researchers at Bauhaus Universitat Weimar in the Prolific Citations Library, a curated collection of research powered by high-quality human data from Prolific.

Papers (1 of 1)

The Viability of Crowdsourcing for RAG Evaluation

Authors: L Gienapp, T Hagen, M Fröbe, M Hagen, B Stein, M Potthast, H Scells

Year: 2025

Published in: ArXiv

Institution: Bauhaus-Universitat Weimar, Friedrich-Schiller-Universitat Jena, Leipzig University, University of Kassel, ScaDS.AI, hessian.AI

Research Area: Crowdsourcing, RAG Evaluation, Artificial Intelligence, AI Evaluation, RAG

Discipline: Artificial Intelligence

The study investigates the feasibility of using crowdsourcing for RAG evaluation, finding that human pairwise judgments are reliable and cost-effective compared to LLM-based or automated methods.

Methods: Two complementary studies on response writing and response utility judgment using 903 human-written and 903 LLM-generated responses for 301 topics; pairwise judgments across seven utility dimensions were collected via human and LLM evaluators.

Key Findings: Human effectiveness in writing and judging responses in RAG scenarios, considering discourse styles and utility dimensions like coverage and coherence.

Citations: 4

Sample Size: 903