University Of Delaware: Research Institution — Prolific Citations Library

Browse 1 peer-reviewed paper from University Of Delaware spanning Crowdsourcing, Data Quality (2025). Research powered by Prolific's high-quality participant data.

This page lists 1 peer-reviewed paper from researchers at University Of Delaware in the Prolific Citations Library, a curated collection of research powered by high-quality human data from Prolific.

Papers (1 of 1)

Data quality in crowdsourcing and spamming behavior detection

Authors: Y Ba, MV Mancenido, EK Chiou, R Pan

Year: 2025

Published in: Behavior Research Methods, 2025 - Springer

Institution: University of Delaware, National Taiwan University, University of British Columbia, Monash University

Research Area: Crowdsourcing, Data Quality, Spamming Behavior Detection, LLM Applications in Behavioral Research

Discipline: Computer Science, Artificial Intelligence, LLM

The paper introduces a systematic method to evaluate crowdsourced data quality and detect spam behaviors through variance decomposition, proposing a spammer index and credibility metrics to improve consistency and reliability in labeling tasks.

Methods: Variance decomposition, Markov chain models, and generalized random effects models were used to assess annotator consistency and credibility; metrics were applied to both simulated and real-world data from two crowdsourcing platforms.

Key Findings: Quality of crowdsourced data, spammer behaviors, annotators’ consistency, and credibility.

Citations: 2