RiskRAG: A Data-Driven Solution for Improved AI Model Risk Reporting

Authors: Pooja S. B. Rao, Sanja Šćepanović, Ke Zhou, Edyta Paulina Bogucka, D Quercia

Published: 2025

Publication: ArXiv

RiskRAG improves AI model risk reporting by offering pre-populated, contextualized risk reports that are preferred by developers, designers, and media professionals over standard model cards.

Methods: Developed a Retrieval Augmented Generation system based on five design requirements co-created with 16 developers, using a dataset of 450K model cards and 600 real-world incidents. Evaluated RiskRAG in preliminary and final studies with a total of 125 participants.

Key Findings: Effectiveness of RiskRAG in improving risk reporting and decision-making compared to standard model cards.

Limitations: Lack of clarity on long-term adoption outside controlled studies and scalability of the approach for new types of models or unforeseen risks.

Institution: Nokia Bell Labs, University of Lausanne

Research Area: AI Risk Management, Model Risk Reporting, RAG Pipeline, RAG

Discipline: Artificial Intelligence

Sample Size: 125 participants