Large language model in creative work: The role of collaboration modality and user expertise

Authors: Z Chen, J Chan

Published: 2024

Publication: Management Science, 2024 - pubsonline.informs.org

Using large language models (LLMs) as sounding boards improves ad content quality for nonexpert users, while using LLMs as ghostwriters can negatively impact expert users due to anchoring effects.

Methods: An experiment comparing ad copy creation with and without LLM assistance, focusing on two collaboration modalities: ghostwriting and sounding board approaches. Ad performance was measured via social media click rates, supported by textual analysis.

Key Findings: Effectiveness of LLM collaboration modalities (ghostwriting vs. sounding board) on ad quality and business outcomes for expert and nonexpert users.

Limitations: Potential generalization to other tasks or industries is unclear; results are specific to ad copy creation and do not account for broader contexts of LLM usage.

Institution: University of Texas Dallas

Research Area: Human-AI Interaction, Creative Work, Behavioral Science

Discipline: Social Science

Citations: 180

DOI: https://doi.org/10.1287/mnsc.2023.03014