Auditing multimodal large language models for context-aware content moderation

Abstract

Multimodal generative artificial intelligence models that can process text, images, and other media provide a context-aware approach to content moderation that may help to address existing biases in these systems. This study proposes conjoint analysis as a methodology to audit these models. We investigate how GPT-4o, a state-of-the-art model, uses demographics and other contextual information when flagging potential hate speech. The results show that the model attends to context similarly to human annotators. In particular, the race of the author is taken into account when deciding whether the use of racist language or reclaimed slurs violates a content policy. Prompting can enhance contextual awareness, although some demographic bias persists. Models relying solely on visual or written identity signals show significantly less contextual variation, highlighting the value of multimodal approaches to automated content moderation. These findings have significant implications for designing more effective moderation systems and methodologies for auditing algorithmic systems.