Human-AI Collaboration in High-Stakes Decision-Making: Work in Progress

Abstract

This work in progress investigates human interaction with an LLM-powered chatbot, presented as either a fellow human or a transparently disclosed AI collaborator, in a high-stakes decision-making simulation—the NASA Moon Survival Task. We will employ a one-way between-subjects design to examine how individuals’ collaboration and communication are influenced by the identity of their partner (AI vs. human). Specifically, we will evaluate individuals’ collaboration processes (i.e., collaborative behaviour and communicative dynamics) and outcomes, alongside their retrospective interaction experience and perceptions of the partner. We will also examine dyadic-level linguistic coordination during the interaction and conduct user profiling to uncover variations in AI collaborative benefits. We anticipate that this study will have four key impacts: safeguarding human-AI collaboration, democratising AI benefits, guiding model improvement, and making methodological contributions. The anonymised dialogues and associated data will be open-sourced upon study completion.