Browse 32 peer-reviewed papers in Replication. Discover studies powered by high-quality human data from Prolific.
This page lists 32 peer-reviewed papers tagged with Replication in the Prolific Citations Library, a curated collection of research powered by high-quality human data from Prolific.
-
Authors: N Byrd
Year: 2025
Published in: Byrd, N. (2025). Reflection-Philosophy Order Effects and Correlations Across Samples. Analysis. DOI: 10.1093/analys/anaf015. https://osf.io/preprints/psyarxiv/y8sdm
Institution: Stevens Institute of Technology
Research Area: Behavioral Research Methods, Experimental Psychology, Crowdsourcing Platforms
Discipline: Psychology
Reflective reasoning correlates with certain philosophical decisions, and the study suggests bidirectional causal paths between reflection and philosophy, with test order effects influencing reflection test outcomes but not philosophical decisions.
Methods: Participants from four sources (Amazon Mechanical Turk, CloudResearch, Prolific, and a university) were tested on reflective reasoning and their decisions on 10 philosophical thought experiments.
Key Findings: Impact of reflective reasoning on philosophical decisions and the effect of test order on reflection and philosophy outcomes.
Citations: 4
-
Authors: HCB Huang
Year: 2025
Published in: Journal of Experimental Psychology: General, 2025 - psycnet.apa.org
Institution: University of British Columbia
Research Area: Human-AI Collaboration, Creativity, Experimental Psychology
Discipline: Experimental Psychology
Moderate levels of human-AI collaboration enhance creative performance due to increased knowledge diversity, but excessive or minimal involvement diminishes this effect.
Methods: Two experiments assigned 139 business professionals and 319 working adults to collaborate with ChatGPT at varying levels, and a follow-up survey among 188 creative industry workers was conducted to replicate findings.
Key Findings: The impact of varying degrees of human-AI collaboration on creative performance, evaluated by human judges, entrepreneurs, and AI metrics.
Citations: 3
Sample Size: 646
-
Authors: M Brassil, É Duncan, C Greene, B Mac Síthigh
Year: 2025
Published in: 2025 - osf.io
Institution: University College Dublin
Research Area: Eyewitness Memory, Misinformation Effect, Behavioral Research Methods, Online Data Collection Platforms
Discipline: Psychology
The study found that data collection contexts significantly influence susceptibility to eyewitness misinformation, with Prolific participants being less accurate and more susceptible compared to Laboratory or general online participants.
Methods: Two studies were conducted comparing eyewitness misinformation susceptibility across Laboratory, Prolific, and General Online participant groups under varying visual perceptual load conditions.
Key Findings: Eyewitness misinformation susceptibility and recall accuracy across Laboratory, Prolific, and General Online participant groups; the effect of visual perceptual load on recall accuracy.
Citations: 1
-
Authors: D Jordan, T Ollerenshaw, A Trexler
Year: 2025
Published in: 2025 - weekendu.uh.edu
Institution: University of Houston, Duke University
Research Area: Experimental Survey Research Methodology
Discipline: Social Science, Research Methodology
Repeated measure designs offer enhanced precision with minimal bias, suitable for various experiments despite slight attenuation of treatment effects.
Methods: Experimentally manipulated six classic political science experiments across three sample types, including extensions with proximity manipulation and sample-type variations.
Key Findings: Suitability and precision of repeated measure designs in survey experiments, including treatment effect estimations and design applicability across different sample types and methodologies.
Citations: 1
Sample Size: 13163
-
Authors: C Rastogi, TH Teh, P Mishra, R Patel, D Wang, M Díaz, A Parrish, AM Davani, Z Ashwood
Year: 2025
Published in: arXiv preprint arXiv:2507.13383, 2025•arxiv.org
Institution: Google DeepMind, Google Research, Google
Research Area: AI alignment, safety evaluation, AI Safety, Multimodal evaluation, Human-AI Interaction, Large Language Models
Discipline: Computer Science, Machine Learning, Artificial Intelligence
This research introduces the DIVE dataset to enable pluralistic alignment in text-to-image models by accounting for diverse safety perspectives, revealing demographic variations in harm perception and advancing T2I model alignment strategies.
Methods: The study involved collecting feedback across 1000 prompts from demographically intersectional human raters to capture diverse safety perspectives, with an emphasis on empirical and contextual differences in harm perception.
Key Findings: Safety perceptions of text-to-image (T2I) model outputs from diverse demographic viewpoints and the influence of these perspectives on alignment strategies.
Citations: 1
Sample Size: 1000
-
Authors: L Woodley, X Roberts-Gaal, R Calcott, F Cushman
Year: 2025
Published in: files.osf.io
Institution: Harvard University
Research Area: Experimental Psychology, Research Methodology, Replication Studies
Discipline: Psychology, Social Science
Explicit demand cues do not alter participant behavior, judgments, or attitudes in online psychology experiments, despite participants adjusting their beliefs about study hypotheses.
Methods: Three preregistered experiments on Prolific tested the impact of explicit demand cues on participant behavior using a dictator game, a moral dilemma vignette, and a group attitude intervention. Participants were randomly assigned to receive information about the study hypothesis or no information.
Key Findings: Whether explicit demand cues influence behavior, judgments, or attitudes in online psychology studies.
Sample Size: 2254
-
Authors: L Lanz, R Briker, FH Gerpott
Year: 2024
Published in: Journal of Business Ethics, 2024 - Springer
Institution: University of Lausanne, University of Neuchâtel, University of Bern
Research Area: AI Ethics, Organizational Behavior, Supervisory Influence in the Workplace
Discipline: Business Ethics, Organizational Behavior, AI Ethics
Employees are less likely to adhere to unethical instructions from AI supervisors compared to human supervisors, partly due to perceived differences in 'mind' and individual characteristics like compliance tendency and age.
Methods: The study employed four experiments using causal forest and transformer-based machine learning algorithms, as well as pre-registered experimental manipulations to evaluate employee behavior towards unethical instructions from AI and human supervisors.
Key Findings: Adherence to unethical instructions from AI versus human supervisors; mediating role of perceived mind and moderating factors like compliance tendency and age.
DOI: https://doi.org/10.1007/s10551-023-05393-1
Citations: 72
Sample Size: 1701
-
Authors: L Cheng, A Chouldechova
Year: 2024
Published in: Proceedings of the 2023 CHI Conference ..., 2023 - dl.acm.org
Institution: Carnegie Mellon University
Research Area: Human-Computer Interaction, Algorithm Aversion, Decision Science
Discipline: Human-Computer Interaction
Giving users process control by selecting the training algorithm mitigates algorithm aversion, but not by changing input factors, while combined outcome and process control is not more effective than each individually.
Methods: Replication study on outcome control and novel process control conditions tested on MTurk and Prolific platforms.
Key Findings: Impact of outcome control, process control, and combined controls on algorithm aversion mitigation.
DOI: https://doi.org/10.1145/3544548.3581253
Citations: 41
-
Authors: Z Cui, N Li, H Zhou
Year: 2024
Published in: A Large-Scale Replication of Psychological ..., 2024 - papers.ssrn.com
Institution: Harbin Institute of Technology at Weihai
Research Area: LLM replication of psychological experiments, Social Science Research Methods, Artificial Intelligence, Psychology
Discipline: Psychological Science
Large Language Models (LLMs) like GPT-4 successfully replicate 76% of main effects and 47% of interaction effects from 154 psychological experiments, but exhibit overestimation and potential false positives, highlighting their complementary role rather than full replacement of human subjects.
Methods: Replication of 154 psychological experiments from top social science journals using GPT-4 as a simulated participant to measure main effects and interaction effects.
Key Findings: The ability of GPT-4 to replicate human responses in psychological experiments and the extent to which it produces similar results in terms of effect direction, significance, and confidence intervals.
Citations: 29
Sample Size: 154
-
Authors: Y Gao, D Lee, G Burtch, S Fazelpour
Year: 2024
Published in: arXiv preprint arXiv:2410.19599, 2024 - arxiv.org
Institution: Boston University, Northeastern University
Research Area: LLMs as Human Surrogates, Social Science Research Methods, Human Behavior Simulation
Discipline: Economics, Artificial Intelligence, Social Science
LLMs fail to accurately replicate human behavior in the 11-20 money request game, cautioning against their use as surrogates for human cognition in social science research.
Methods: The study evaluates the reasoning depth of various advanced LLMs through their performance on the 11-20 money request game, analyzing failure points related to input language, roles, and safeguarding.
Key Findings: The ability of LLMs to replicate human-like behavior and reasoning distribution in the context of social science simulations.
Citations: 25
-
Authors: S Valentin, S Kleinegesse, NR Bramley, P Seriès
Year: 2024
Published in: Elife, 2024 - elifesciences.org
Institution: University of Edinburgh, University of Cambridge
Research Area: Bayesian Optimal Experimental Design (BOED) in Behavioral Research
Discipline: Artificial Intelligence, Psychology
The paper presents a tutorial on using Bayesian optimal experimental design (BOED) and machine learning to design experiments that efficiently test and evaluate cognitive models, validated via simulations and a real-world case study of exploration-exploitation decision-making.
Methods: The paper employs Bayesian optimal experimental design (BOED) coupled with machine learning to identify optimal experimental configurations. Simulations and a real-world multi-armed bandit experiment are used for validation.
Key Findings: The capacity of BOED to distinguish between cognitive models, parameters explaining human behavior, and how people balance exploration and exploitation.
DOI: https://doi.org/10.7554/eLife.86224
Citations: 15
-
Authors: V Cheung, M Maier, F Lieder
Year: 2024
Published in: Psyarxiv preprint, 2024 - files.osf.io
Institution: University College LondonA
Research Area: AI Ethics, Moral Decision-Making, Cognitive Biases in LLMs, AI Bias
Discipline: Artificial Intelligence, Ethics
Citations: 11
-
Authors: V Kewenig, C Edwards
Year: 2024
Published in: ... and Rechardt, Akilles ..., 2023 - papers.ssrn.com
Research Area: Multimodal AI, Cognitive Science, Visual-Linguistic Integration
Discipline: Artificial Intelligence, Computational Linguistics, Cognitive Science
Citations: 2
-
Authors: A Welivita, P Pu
Year: 2024
Published in: ArXiv
Institution: École Polytechnique Fédérale de Lausanne
Research Area: Large Language Models, Empathy, Human-AI Interaction
Discipline: Artificial Intelligence, Human-Computer Interaction, Social Science
-
Authors: E Jahani, B Manning, J Zhang, H TuYe, M Alsobay, C Nicolaides, S Suri, D Holtz
Year: 2024
Published in: ArXiv
Institution: Massachusetts Institute of Technology, Microsoft Research, Stanford University, University of California Berkeley, University of Cyprus, University of Maryland
Research Area: Human-AI Interaction, Generative AI, Prompt Engineering
Discipline: Artificial Intelligence, focusing on Human-AI Interaction, Generative AI
-
Authors: J de Winter, D Dodou, YB Eisma
Year: 2024
Published in: Discover, 2024 - repository.tudelft.nl
Institution: Delft University of Technology
Research Area: Psychology
Discipline: Psychology
-
Authors: Winnie Bahati Mbaka, Katja Tuma
Year: 2024
Published in: ArXiv
Institution: Vrje Universiteit Amsterdam
Research Area: Security Threat Validation, Data Flow Diagrams, AI Cybersecurity
Discipline: Artificial Intelligence
-
Authors: N Gagné, L Franzen
Year: 2023
Published in: ... Open: the official journal of the ..., 2023 - swisspsychologyopen.com
Institution: Concordia University, University of Lübeck
Research Area: Cognitive Psychology, Neuroscience, Behavioral Research Methods
Discipline: Cognitive Psychology, Neuroscience
Citations: 75
-
Authors: LD Griffin, B Kleinberg, M Mozes, KT Mai, M Vau
Year: 2023
Published in: arXiv preprint arXiv ..., 2023 - arxiv.org
Institution: University College London, Tilburg University
Research Area: LLM Influence, Psychology, Mental Health Research, Large Language Models
Discipline: Artificial Intelligence, Psychology
Citations: 30
-
Authors: L Griffin, B Kleinberg, M Mozes, K Mai
Year: 2023
Published in: Proceedings of the ..., 2023 - aclanthology.org
Institution: University College London, Tilburg University
Research Area: LLM Influence and Persuasion, Large Language Models
Discipline: Social Science
Citations: 25
