Browse 9 peer-reviewed papers in With Dataset. Discover studies powered by high-quality human data from Prolific.
This page lists 9 peer-reviewed papers tagged with With Dataset in the Prolific Citations Library, a curated collection of research powered by high-quality human data from Prolific.
-
Authors: J Geng, J Tonglet, I Gurevych
Year: 2026
Published in: arXiv preprint arXiv:2510.23508, 2025•arxiv.org
Institution: KU Leuven, TU Darmstadt, Ubiquitous Knowledge Processing Lab, MBZUAI, ATHENE
Research Area: Human-Computer Interaction
Discipline: Machine Learning, Artificial Intelligence
M4FC is a new dataset that addresses limitations in existing multimodal fact-checking datasets by providing multilingual and multicultural claims verified by professional fact-checkers across six fact-checking tasks.
Methods: The dataset was created by pairing 4,982 images with 6,980 claims, which were verified by professional fact-checkers from 22 organizations covering diverse cultural and geographic contexts. The claims are available in up to ten languages and span six different multimodal fact-checking tasks.
Key Findings: The study measured the efficacy of the M4FC dataset across six multimodal fact-checking tasks, with a focus on how combining intermediate tasks affects the performance of verdict prediction.
Citations: 3
Sample Size: 6980
-
Authors: W Liu, X Mou, H Yan, Z, Wei, Y He
Year: 2026
Published in: arXiv preprint arXiv:2606.04075, 2026•arxiv.org
Institution: King’s College London, Fudan University, Shanghai Innovation Institute, The Alan Turing Institute
Research Area: Human-Computer Interaction
Discipline: Machine Learning, Artificial Intelligence
The paper finds that large language models can exploit gaps in societal rules, leading to regulatory loophole discovery, necessitating a new post-training approach for safely integrating LLMs into society.
Methods: The study introduced the SocioHack sandbox, consisting of 72 societal environments, to investigate reward hacking and loophole discovery by LLMs.
Key Findings: The study measured the emergence of reward hacking in societal environments and the ability of models to find and exploit loopholes in social rules.
Sample Size: 72
-
Authors: C Yuan, B Ma, Z Zhang, B Prenkaj, F Kreuter, G Kasneci
Year: 2026
Published in: arXiv preprint arXiv:2601.08634, 2026•arxiv.org
Institution: Munich Center for Machine Learning, LMU Munich, Technical University of Munich
Research Area: Artificial Intelligence, AI Ethics, AI Alignment, Political Science, Computational Social Science
Discipline: Computer Science, Natural Language Processing
This paper examines how large language models’ (LLMs) political outputs shift when you explicitly prime them with different moral values. Instead of just assigning fake personas (like “pretend to be liberal”), the authors condition models to endorse or reject specific moral values (e.g., utilitarianism, fairness, authority). They then measure how those moral primes move the models’ positions in...
DOI: https://doi.org/10.48550/arXiv.2601.08634
-
Authors: N Petrova, A Gordon, E Blindow
Year: 2026
Published in: Open review
Institution: Prolific
Research Area: Human-centered AI evaluation, Bayesian statistics, Responsible AI, AI alignment, LLM Evaluation
Discipline: Machine Learning, Artificial Intelligence
The study introduces HUMAINE, a multidimensional evaluation framework for LLMs, revealing demographic-specific preference variations and ranking google/gemini-2.5-pro as the top-performing model with a posterior probability of 95.6%.
Methods: Multi-turn naturalistic conversations analyzed using a hierarchical Bayesian Bradley-Terry-Davidson model with post-stratification to census data, stratified across 22 demographic groups.
Key Findings: Performance of 28 LLMs across five human-centric dimensions, accounting for demographic-specific preferences.
Sample Size: 23404
-
Authors: T Hu, N Collier
Year: 2025
Published in: arXiv preprint arXiv:2503.03335, 2025 - arxiv.org
Institution: University of Cambridge
Research Area: Affective Computing, Natural Language Processing, Computational Social Science
Discipline: Computational Social Science
The iNews dataset is a multimodal resource for studying personalized affective responses to news, improving modeling accuracy by incorporating annotator persona metadata.
Methods: 292 demographically diverse UK participants annotated 2,899 Facebook news posts with multidimensional labels (e.g., emotions, valence, arousal), combined with comprehensive participant persona data.
Key Findings: Modeled personalized affective responses to news through annotations capturing valence, arousal, emotions, and persona metadata.
Citations: 2
Sample Size: 2899
-
Authors: HR Kirk, M Bartolo, A Whitefield, P Rottger
Year: 2024
Published in: Advances in ..., 2024 - proceedings.neurips.cc
Institution: Meta, Cohere, AWS AI Labs, Contextual AI, Factored AI, University of Oxford, Bocconi University, Meedan, Hugging Face, University College London, ML Commons, University of Pennsylvania
Research Area: LLM Alignment, Human Feedback, Multicultural Studies
Discipline: Artificial Intelligence, Computational Social Science
The PRISM Alignment Dataset presents a large-scale, culturally diverse human feedback dataset linking sociodemographic profiles of 1,500 participants from 75 countries to their contextual preferences and fine‑grained ratings in 8,011 live conversations with 21 LLMs. This enables analysis of how subjective values vary across people and cultures in LLM alignment data.
DOI: https://doi.org/10.52202/079017-3342
Citations: 204
-
Authors: M Ku, T Li, K Zhang, Y Lu, X Fu, W Zhuang
Year: 2024
Published in: - arXiv preprint arXiv …, 2023 - arxiv.org
Institution: University of Waterloo, Ohio State University, University of California Santa Barbara, University of Pensylvania
Research Area: AI alignment, Representation learning, Cognitive computational modeling, Vision foundation models evaluation, Multimodal, Vision models
Discipline: Computer Science, Artificial Intelligence, Machine Learning
This paper presents a method for **aligning machine vision model representations with human visual similarity judgments across different abstraction levels, improving how well models reflect human perceptual and conceptual organization and enhancing generalization and uncertainty prediction.
DOI: https://doi.org/10.48550/arXiv.2310.01596
Citations: 59
-
Authors: J Zou, K Vodrahalli
Year: 2024
Published in: ArXiv
Institution: Stanford University
Research Area: Human-AI Interaction in Artistic Creations
Discipline: Artificial Intelligence, Human-Computer Interaction
Citations: 12
-
Authors: Mahjabin Nahar, Haeseung Seo, Eun-Ju Lee, Aiping Xiong, Dongwon Lee
Year: 2024
Published in: ArXiv
Institution: Pennsylvania State University, Seoul National University
Research Area: LLM Hallucinations, Human Perception, Warning Effects in HCI
Discipline: Artificial Intelligence, Human-Computer Interaction
