Authors: P. Schoenegger, F. Salvi, J. Liu, X. Nan, R. Debnath, B. Fasolo, E. Leivada, G. Recchia, F. Günther, A. Zarifhonarvar, J. Kwon, Z. Ul Islam, M. Dehnert, D. Y. H. Lee, M. G. Reinecke, D. G. Kamper, M. Kobaş, A. Sandford, J. Kgomo, L. Hewitt, S. Kapoor, K. Oktar, E. E. Kucuk, B. Feng, C. R. Jones, I. Gainsburg, S. Olschewski, N. Heinzelmann, F. Cruz, B. M. Tappin, T. Ma, P. S. Park, R. Onyonka, A. Hjorth, P. Slattery, Q. Zeng, L. Finke, I. Grossmann, A. Salatiello, E. Karger
Year: 2025
Published in: arXiv preprint arXiv ..., 2025 - arxiv.org
Institution: London School of Economics and Political Science, University of Cambridge, University College London, Massachusetts Institute of Technology, University of Oxford, Modulo Research, Stanford University, Federal Reserve Bank of Chicago, ETH Zürich, University of Johannesburg
Research Area: Natural Language Processing
Discipline: Social Science, Artificial Intelligence
This paper compares a frontier LLM (Claude Sonnet 3.5) against incentivized human persuaders in a conversational quiz setting, finding that the AI's persuasion capabilities surpass those of humans with real-money bonuses tied to performance.
Citations: 16
Authors: S Kapoor, N Gruver, M Roberts
Year: 2024
Published in: Advances in ..., 2024 - proceedings.neurips.cc
Institution: Abacus AI, University of Cambridge, New York University, Columbia University
Research Area: Uncertainty Estimation, LLM Limitations, Know-What-You-Don't-Know, Computational Cognition
Discipline: Artificial Intelligence
Fine-tuning large language models (LLMs) on a small dataset of graded examples improves uncertainty estimations, enhancing their applicability in high-stakes scenarios and human-AI collaboration.
Methods: The researchers fine-tuned LLMs using a small dataset of graded correct and incorrect answers with LoRA (Low-Rank Adaptation) to create uncertainty estimates and conducted a user study to investigate their utility in human-AI collaboration.
Key Findings: Calibration and generalization of uncertainty estimates, performance of fine-tuning LLMs for uncertainty estimation, and human-AI interaction improvements informed by uncertainty data.
Citations: 71
Sample Size: 1000
