Collaborating with ai agents: Field experiments on teamwork, productivity, and performance

Abstract

To uncover how AI agents change productivity, performance, and work processes, we introduce Pairit -- an experimentation platform enabling humans and AI agents to collaborate in integrative workspaces. In a large-scale marketing experiment on the platform, 2310 participants were randomly assigned to human-human and human-AI teams. The teams exchanged 183,691 messages and created 63,656 image edits, 1,960,095 ad copy edits, and 10,375 AI-generated images while producing 11,138 ads for a large think tank. Analysis of fine-grained communication, collaboration, and workflow logs revealed that collaborating with AI agents increased communication by 63% and allowed humans to engage in 71% less direct text editing. While human-AI teams engaged in 18% more process and content communication, human-human teams engaged in 29% more social and emotional communication. Humans in human-AI teams experienced 73% greater productivity per worker and produced higher-quality ad copy, while human-human teams produced higher-quality images, suggesting AI agents require fine-tuning for multimodal workflows. Field tests of the ad campaigns accumulated ~5M ad impressions and revealed that ads with higher image quality (produced by human-human collaborations) and higher text quality (produced by human-AI collaborations) performed significantly better on click-through rates, view through rates, and cost per click metrics. Together, these results suggest that human collaboration with AI agents significantly reshapes communication patterns and work processes and increases productivity, while improving some dimensions of output quality and deteriorating others. We hope the release of the extensible Pairit platform will accelerate RCTs of human-AI collaboration across a variety of work tasks and contexts.