Participant Interactions with Artificial Intelligence: Using Large Language Models to Generate Research Materials for Surveys and Experiments
Authors: TS Behrend, RN Landers
Published: 2025
Publication: Journal of Business and Psychology, 2025 - Springer
The paper proposes a framework with five use cases for integrating large language models into survey and experimental research, introduces the Qualtrics-AI Link (QUAIL) tool, and highlights technical and ethical considerations for using LLMs effectively and validly.
Methods: The paper outlines a decision-making framework for five potential uses of LLMs in survey and experimental design, introduces software (QUAIL) for integrating LLM knowledge into Qualtrics, and details technical steps such as prompt engineering, model testing, and validity monitoring.
Key Findings: Applications, implementation strategies, and ethical considerations of large language models in psychological research material development.
Limitations: Guidance is dependent on researchers' expertise with both LLMs and experimental design; external validity risks and ethical challenges require careful consideration.
Institution: University of Nebraska-Lincoln, University of Minnesota
Research Area: LLM in Behavioral Science Research, AI-Assisted Research Methodology
Discipline: Behavioral Science, Psychology, Artificial Intelligence
Citations: 6
DOI: https://doi.org/10.1007/s10869-025-10035-6