Authors: N Tyulina, Y Yu, TA Emmanouil, SI Levitan
Year: 2025
Published in: Proceedings of the 7th ACM ..., 2025 - dl.acm.org
Institution: University of Cambridge, University of Bath, University of Edinburgh, New York University
Research Area: Human-AI Interaction, Trust and Perception, Nonverbal Communication
Discipline: Applied Linguistics
Trust judgments are primarily influenced by auditory cues in both humans and multimodal models, though subtle differences in modality weighting exist between them.
Methods: Behavioral experiment with trust ratings of bimodal stimuli across four trust congruence conditions, combined with a multimodal model trained using HuBERT and ResNet-50 with late fusion, analyzed using Permutation Feature Importance (PFI).
Key Findings: The construction of trust from visual and auditory signals in both humans and multimodal models, focusing on modality dominance and feature weighting.
Sample Size: 150