Data Quality: Research Area — Prolific Citations Library

Discover 8 peer-reviewed studies in Data Quality (2021–2025). Explore research findings powered by Prolific's diverse participant panel.

This page lists 8 peer-reviewed papers in the research area of Data Quality in the Prolific Citations Library, a curated collection of research powered by high-quality human data from Prolific.

Papers (8 of 8)

Data quality in crowdsourcing and spamming behavior detection

Authors: Y Ba, MV Mancenido, EK Chiou, R Pan

Year: 2025

Published in: Behavior Research Methods, 2025 - Springer

Institution: University of Delaware, National Taiwan University, University of British Columbia, Monash University

Research Area: Crowdsourcing, Data Quality, Spamming Behavior Detection, LLM Applications in Behavioral Research

Discipline: Computer Science, Artificial Intelligence, LLM

The paper introduces a systematic method to evaluate crowdsourced data quality and detect spam behaviors through variance decomposition, proposing a spammer index and credibility metrics to improve consistency and reliability in labeling tasks.

Methods: Variance decomposition, Markov chain models, and generalized random effects models were used to assess annotator consistency and credibility; metrics were applied to both simulated and real-world data from two crowdsourcing platforms.

Key Findings: Quality of crowdsourced data, spammer behaviors, annotators’ consistency, and credibility.

Citations: 2
Detecting the corruption of online questionnaires by artificial intelligence

Authors: B Lebrun, S Temtsin, A Vonasch

Year: 2024

Published in: Frontiers in Robotics and ..., 2024 - frontiersin.org

Institution: University of Lausanne, University of California Berkeley, University of Massachusetts Amherst, Arizona State University

Research Area: AI in Social Science Research, Survey Methodology, Data Quality

Discipline: Artificial Intelligence

The study examines the integrity of online questionnaire responses and concludes that humans can identify AI-generated text with 76% accuracy, but current AI detection systems are ineffective, raising concerns about data quality in online surveys.

Methods: Human participants and automatic AI detection systems were tested on their ability to differentiate AI-generated text from human-generated text in the context of online questionnaires.

Key Findings: The study measured the ability of humans and AI detection tools to correctly identify whether text was generated by a human or an AI system for online questionnaire responses.

DOI: https://doi.org/10.3389/frobt.2023.1277635

Citations: 26
Comparing data quality from an online and in-person lab sample on dynamic theory of mind tasks

Authors: AC Krendl, K Hugenberg, DP Kennedy

Year: 2024

Published in: Behavior research methods, 2024 - Springer

Institution: Indiana University

Research Area: Psychological Research Methods, Data Quality in Online Experiments, Theory of Mind Assessment

Discipline: Research Methodology, Cognitive Psychology

This study found that online samples can reliably complete dynamic, complex theory of mind tasks, though familiarity with task content can influence performance.

Methods: Compared in-lab and online participants' performance on two dynamic theory of mind tasks, using one familiar and one relatively novel TV-based paradigm and counterbalancing task order.

Key Findings: Performance on theory of mind tasks, including inferring beliefs, understanding motivations, detecting deception, identifying faux pas, and understanding emotions.

Citations: 13

Sample Size: 668
Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA

Authors: BD Douglas, PJ Ewell, M Brauer

Year: 2023

Published in: Plos one, 2023 - journals.plos.org

Institution: University of Alabama, University of Wisconsin-Madison, Florida Atlantic University

Research Area: Social Science Research Methods, Behavioral Research, Data Quality in Crowdsourcing

Discipline: Social Science Research Methods

Citations: 1598
Inattentive responding can induce spurious associations between task behaviour and symptom measures

Authors: S Zorowitz, J Solis, Y Niv, D Bennett

Year: 2023

Published in: Nature human behaviour, 2023 - nature.com

Institution: Princeton University, Rutgers University, Monash University

Research Area: Research Methodology, Behavioral Research, Experimental Psychology (focus on data quality and spurious correlations)

Discipline: Behavioral Science

DOI: https://doi.org/10.1038/s41562-023-01640-7

Citations: 110
Data quality of platforms and panels for online behavioral research

Authors: E Peer, D Rothschild, A Gordon, E Damer

Year: 2022

Published in: Behavior Research Methods, 2022 - Springer

Institution: The Hebrew University of Jerusalem, Microsoft Research, Prolific

Research Area: Online Behavioral Research, Data Quality, Research Methods

Discipline: Computational Social Science, Behavioral Research

Citations: 2112
Reply to MTurk, Prolific or panels? Choosing the right audience for online research

Authors: L Litman, A Moss, C Rosenzweig

Year: 2021

Published in: Choosing the right ..., 2021 - papers.ssrn.com

Institution: Prolific

Research Area: Online Research Methods, Crowdsourcing Platforms, Data Quality, Participant Recruitment

Discipline: Computational Social Science, Behavioral Research Methods

Citations: 133
Online data collection in auditory perception and cognition research: Recruitment, testing, data quality and ethical considerations

Authors: T Eerola, J Armitage, N Lavan

Year: 2021

Published in: Auditory Perception & ..., 2021 - Taylor & Francis

Institution: Durham University

Research Area: Auditory Perception and Cognition, Research Methodology, Data Quality, Ethics in Online Research

Discipline: Cognitive Psychology, Research Methodology

Citations: 54