Improving annotation quality: empirical insights into bias, human-AI collaboration, and workflow design

Authors: J Beck

Published: 2025

Publication: 2025 - edoc.ub.uni-muenchen.de

The study empirically evaluates annotation bias, proposes strategies to reduce its impact, and explores the use of large language models in automated and hybrid annotation workflows.

Methods: Empirical assessments and experimental evaluations involving annotation workflows and large language models.

Key Findings: Annotation bias, annotation quality, and the effectiveness of hybrid workflows integrating human input and AI models.

Institution: Ludwig-Maximilians-Universität München, University of Bayreuth

Research Area: Annotation Quality, Human-AI Collaboration, Behavioral Science, Human-Computer Interaction (HCI)

Discipline: Human-Computer Interaction (HCI)