Browse 17 peer-reviewed papers in Methodology. Discover studies powered by high-quality human data from Prolific.
This page lists 17 peer-reviewed papers classified as Methodology in the Prolific Citations Library, a curated collection of research powered by high-quality human data from Prolific.
-
Authors: H Mohseni, T Kujala, J Silvennoinen
Year: 2026
Published in: SPRINGER
Institution: University of Jyväskylä
Research Area: Migration studies, Social indicators, Psychometrics, Quantitative social science methods
Discipline: Social sciences
Developed and validated a multidimensional place-belongingness scale to assess immigrants' sense of belonging to geographic locations, identifying four factors: feeling at home, accepted, empowered, and secure.
Methods: Survey data from 270 immigrants worldwide analyzed using exploratory factor analysis.
Key Findings: The subjective sense of place-belongingness, decomposed into four factors: feeling at home, feeling accepted, feeling empowered, and feeling secure.
Sample Size: 270
-
Authors: K Dalal, D Koceja, G Hussein, J Xu, Y Zhao, Y Song, S Han, KC Cheung, J Kautz, C Guestrin, T Hashimoto, S Koyejo, Y Choi, Y Sun, X Wang
Year: 2025
Published in: ArXiv
Institution: Nvidia, Stanford University, UT Austin, University of California Berkeley, University of California San Diego
Research Area: Video Generation, Diffusion Models, Test-Time Training
Discipline: Computer Science
The paper introduces Test-Time Training (TTT) layers into Transformers to generate coherent one-minute videos from text storyboards, outperforming baselines in storytelling coherence but facing efficiency and artifact challenges.
Methods: Experimentation with Test-Time Training layers embedded in pre-trained Transformer models, evaluated using a dataset curated from Tom and Jerry cartoons and compared against Mamba 2, Gated DeltaNet, and sliding-window attention layers.
Key Findings: Effectiveness of video generation methods in creating coherent multi-scene stories in one-minute videos.
Citations: 52
Sample Size: 100
-
Authors: TS Behrend, RN Landers
Year: 2025
Published in: Journal of Business and Psychology, 2025 - Springer
Institution: University of Nebraska-Lincoln, University of Minnesota
Research Area: LLM in Behavioral Science Research, AI-Assisted Research Methodology
Discipline: Behavioral Science, Psychology, Artificial Intelligence
The paper proposes a framework with five use cases for integrating large language models into survey and experimental research, introduces the Qualtrics-AI Link (QUAIL) tool, and highlights technical and ethical considerations for using LLMs effectively and validly.
Methods: The paper outlines a decision-making framework for five potential uses of LLMs in survey and experimental design, introduces software (QUAIL) for integrating LLM knowledge into Qualtrics, and details technical steps such as prompt engineering, model testing, and validity monitoring.
Key Findings: Applications, implementation strategies, and ethical considerations of large language models in psychological research material development.
DOI: https://doi.org/10.1007/s10869-025-10035-6
Citations: 6
-
Authors: Y Ba, MV Mancenido, EK Chiou, R Pan
Year: 2025
Published in: Behavior Research Methods, 2025 - Springer
Institution: University of Delaware, National Taiwan University, University of British Columbia, Monash University
Research Area: Crowdsourcing, Data Quality, Spamming Behavior Detection, LLM Applications in Behavioral Research
Discipline: Computer Science, Artificial Intelligence, LLM
The paper introduces a systematic method to evaluate crowdsourced data quality and detect spam behaviors through variance decomposition, proposing a spammer index and credibility metrics to improve consistency and reliability in labeling tasks.
Methods: Variance decomposition, Markov chain models, and generalized random effects models were used to assess annotator consistency and credibility; metrics were applied to both simulated and real-world data from two crowdsourcing platforms.
Key Findings: Quality of crowdsourced data, spammer behaviors, annotators’ consistency, and credibility.
Citations: 2
-
Authors: Y Zhang, J Pang, Z Zhu, Y Liu
Year: 2025
Published in: arXiv preprint arXiv:2506.06991, 2025 - arxiv.org
Institution: Rutgers University, University of California Santa Cruz
Research Area: Artificial Intelligence, Computational Social Science
Discipline: Computational Social Science
The paper proposes a training-free scoring mechanism using peer prediction to detect and mitigate LLM-assisted cheating in crowdsourced annotation tasks, with theoretical guarantees and empirical validation.
Methods: A peer prediction-based mechanism quantifies correlations between worker answers while conditioning on LLM-generated labels, without requiring ground truth or high-dimensional training data.
Key Findings: Detection of LLM-assisted low-effort cheating in crowdsourced annotation tasks, focusing on theoretical effectiveness and empirical robustness.
DOI: https://doi.org/10.48550/arXiv.2506.06991
Citations: 1
-
Authors: S Liu, Z Cai, H Wang, Z Ma, X Li
Year: 2025
Published in: arXiv preprint arXiv:2505.19134, 2025 - arxiv.org
Institution: Meta, Imperial College London
Research Area: Artificial Intelligence, Crowdsourcing, LLM
Discipline: Artificial Intelligence
The paper develops a principal-agent model to incentivize high-quality human annotations using golden questions and identifies criteria for these questions to effectively monitor annotators' performance.
Methods: The authors use a principal-agent model with maximum likelihood estimators (MLE) and hypothesis testing to design incentive-compatible systems for annotators. Golden questions of high certainty and similar format to normal data were selected and validated through experiments.
Key Findings: The effectiveness of golden questions for incentivizing and monitoring high-quality human annotations in preference data.
DOI: https://doi.org/10.48550/arXiv.2505.19134
Citations: 1
-
Authors: N Schwitter
Year: 2025
Published in: Social Science Computer Review, 2025 - journals.sagepub.com
Institution: University of Lucerne
Research Area: Artificial Intelligence in Social Science Research Methods, Factorial Survey Experiments, Visual Vignettes Generation
Discipline: Social Science
This paper explores the use of generative AI for creating visual vignettes in factorial survey experiments, highlighting their potential to boost realism and engagement while addressing ethical and technical challenges.
Methods: Techniques for generating and selectively editing AI-generated images were demonstrated, and a pretest with human participants was conducted to evaluate perceptions and interpretations of the images.
Key Findings: Application of AI-generated visual vignettes in social science research and participant interpretation of these images.
Citations: 1
-
Authors: Liudmila Zavolokina, Kilian Sprenkamp, Zoya Katashinskaya, Daniel Gordon Jones
Year: 2025
Published in: ArXiv
Institution: University of Zurich
Research Area: AI Ethics, AI Bias, News Literacy, Critical Thinking, Computational Social Science
Discipline: Computational Social Science
The study explores leveraging inherent biases in AI to enhance critical thinking in news consumption, proposing strategies such as bias awareness, personalization, and gradual introduction of diverse perspectives.
Methods: Qualitative user study investigating user responses to personalized AI-driven propaganda detection tools.
Key Findings: The effectiveness of AI bias-based strategies in improving critical thinking and news readers’ engagement with diverse perspectives.
-
Authors: N Haduong
Year: 2025
Published in: 2025 - search.proquest.com
Institution: University of Washington
Research Area: Human-AI Interaction and Perception
Discipline: Human-Computer Interaction (HCI)
The research focuses on developing methodologies to bridge the gap between controlled laboratory studies and real-world human-AI perceptions and interactions, promoting task immersion and intrinsic motivation to model realistic behaviors.
Methods: Used task immersion techniques, domain-specific recruitment, error taxonomy development, and CPS-TaskForge environment generator for systematic study of collaborative problem solving and AI-assisted decision-making.
Key Findings: Human perceptions of AI in collaborative problem solving, understanding risks in AI-assisted decision making, and user behavior under performance pressure with AI advice.
-
Authors: Paresh Chaudhary, Yancheng Liang, Daphne Chen, Simon S. Du, Natasha Jaques
Year: 2025
Published in: ArXiv
Institution: University of Washington
Research Area: Human-AI Coordination, Zero-Shot Coordination, Adversarial Training, Generative Models
Discipline: Artificial Intelligence, Human-Computer Interaction (HCI)
The paper introduces GOAT, a novel framework combining pretrained generative models and adversarial training to improve human-AI coordination, achieving state-of-the-art performance on the Overcooked benchmark with real human partners.
Methods: The study utilized a frozen pretrained generative model to simulate cooperative agent policies and applied adversarial training to dynamically generate challenging human-AI interaction scenarios for training.
Key Findings: The effectiveness of GOAT in generalizing human-AI coordination strategies and its performance on the Overcooked benchmark.
-
Authors: Elyas Meguellati1, Assad Zeghina2, Shazia Sadiq1, Gianluca Demartini1
Year: 2025
Published in: ArXiv
Institution: University of Queensland, University of Strasbourg
Research Area: Natural Language Processing, Harmful Content Detection
Discipline: Natural Language Processing
The paper introduces an approach using LLM-based semantic augmentation for harmful content detection on social media, achieving performance comparable to human-annotated models but at reduced cost.
Methods: The researchers utilize LLMs to clean noisy text and generate explanations for context-rich preprocessing, then evaluate the augmented training sets on multiple high-context datasets such as SemEval 2024 Persuasive Meme, Google Jigsaw toxic comments, and Facebook hateful memes datasets.
Key Findings: The efficacy of LLM-based semantic augmentation in enhancing training sets for social media tasks such as propaganda detection, hateful meme classification, and toxicity identification.
-
Authors: F Joessel, S Denkinger, PE Joessel, CS Green
Year: 2025
Published in: Acta Psychologica, 2025 - Elsevier
Institution: Max Planck Institute, University of Potsdam, University of Maryland, University of Zurich, University of Arizona
Research Area: Online cognitive training, Automated psychological studies, Crowdsourcing, behavioral research
Discipline: Psychology
The study introduces a fully online method for conducting cognitive training experiments using Prolific, significantly reducing resource demands while achieving robust results and diverse participant recruitment.
Methods: Participants were recruited via Prolific, assigned to groups using a pseudo-randomized procedure, and completed a 12-hour remote cognitive training study with pre- and post-test assessments monitored via custom dashboards.
Key Findings: Impact of a 12-hour cognitive training intervention on participants' cognitive functions, conducted in a remote and automated manner.
-
Authors: Y Wu, C Huang, F Yang, F Wang
Year: 2025
Published in: ArXiv
Institution: Nvidia, National Taiwan University
Research Area: Motion Customization of Text-to-Video Diffusion Models
Discipline: Computer Vision, Pattern Recognition
MotionMatcher is a novel framework for motion customization in text-to-video (T2V) diffusion models, using high-level spatio-temporal motion features rather than pixel-level objectives, achieving state-of-the-art performance.
Methods: Fine-tuning pre-trained text-to-video diffusion models at feature level by comparing spatio-temporal motion features instead of pixel-level objectives to address motion customization from reference videos.
Key Findings: Efficacy of motion customization in T2V models; ability to accurately capture complex motion and avoid content leakage from reference videos.
-
Authors: Pooja S. B. Rao, Sanja Šćepanović, Ke Zhou, Edyta Paulina Bogucka, D Quercia
Year: 2025
Published in: ArXiv
Institution: Nokia Bell Labs, University of Lausanne
Research Area: AI Risk Management, Model Risk Reporting, RAG Pipeline, RAG
Discipline: Artificial Intelligence
RiskRAG improves AI model risk reporting by offering pre-populated, contextualized risk reports that are preferred by developers, designers, and media professionals over standard model cards.
Methods: Developed a Retrieval Augmented Generation system based on five design requirements co-created with 16 developers, using a dataset of 450K model cards and 600 real-world incidents. Evaluated RiskRAG in preliminary and final studies with a total of 125 participants.
Key Findings: Effectiveness of RiskRAG in improving risk reporting and decision-making compared to standard model cards.
Sample Size: 125
-
Authors: Gemma Team
Year: 2024
Published in: ArXiv
Institution: Google DeepMind, Google
Research Area: LLM, Model Efficiency, Architecture
Discipline: Artificial Intelligence
Gemma 2 introduces scalable Transformer-based language models (2B-27B parameters) enhanced with techniques like local-global and group-query attention, achieving state-of-the-art performance for their size and competing with larger models.
Methods: The study applied modifications to the Transformer architecture, such as local-global attentions and group-query attention, as well as knowledge distillation training for select model sizes.
Key Findings: Performance of lightweight language models in terms of efficiency and competitiveness with larger models.
Citations: 1649
-
Authors: L Hewitt, A Ashokkumar, I Ghezae, R Willer
Year: 2024
Published in: Preprint, 2024 - samim.io
Institution: Stanford University, New York University
Research Area: Social Science Experiments, Large Language Model Prediction, LLM
Discipline: Computational Social Science
The study presents a framework using large language models to predict outcomes of social science field experiments, achieving 78% accuracy but facing challenges with experiments on complex social issues.
Methods: Authors used an automated framework powered by large language models to predict outcomes of 276 field experiments drawn from economics literature.
Key Findings: The prediction accuracy of large language models for outcomes of field experiments addressing various human behaviors.
Citations: 68
Sample Size: 276
-
Authors: S Valentin, S Kleinegesse, NR Bramley, P Seriès
Year: 2024
Published in: Elife, 2024 - elifesciences.org
Institution: University of Edinburgh, University of Cambridge
Research Area: Bayesian Optimal Experimental Design (BOED) in Behavioral Research
Discipline: Artificial Intelligence, Psychology
The paper presents a tutorial on using Bayesian optimal experimental design (BOED) and machine learning to design experiments that efficiently test and evaluate cognitive models, validated via simulations and a real-world case study of exploration-exploitation decision-making.
Methods: The paper employs Bayesian optimal experimental design (BOED) coupled with machine learning to identify optimal experimental configurations. Simulations and a real-world multi-armed bandit experiment are used for validation.
Key Findings: The capacity of BOED to distinguish between cognitive models, parameters explaining human behavior, and how people balance exploration and exploitation.
DOI: https://doi.org/10.7554/eLife.86224
Citations: 15