Large language models must be taught to know what they don't know
Authors: S Kapoor, N Gruver, M Roberts
Published: 2024
Publication: Advances in ..., 2024 - proceedings.neurips.cc
Fine-tuning large language models (LLMs) on a small dataset of graded examples improves uncertainty estimations, enhancing their applicability in high-stakes scenarios and human-AI collaboration.
Methods: The researchers fine-tuned LLMs using a small dataset of graded correct and incorrect answers with LoRA (Low-Rank Adaptation) to create uncertainty estimates and conducted a user study to investigate their utility in human-AI collaboration.
Key Findings: Calibration and generalization of uncertainty estimates, performance of fine-tuning LLMs for uncertainty estimation, and human-AI interaction improvements informed by uncertainty data.
Limitations: The study relied on a relatively small dataset for fine-tuning, and the scalability or applicability to closed-source models and larger datasets was not examined in depth.
Institution: Abacus AI, University of Cambridge, New York University, Columbia University
Research Area: Uncertainty Estimation, LLM Limitations, Know-What-You-Don't-Know, Computational Cognition
Discipline: Artificial Intelligence
Sample Size: 1000 participants
Citations: 71