Poster Thu, Apr 23, 2026 • 6:30 AM – 9:00 AM PDT Pavilion 4 P4-#5107

AudioTrust: Benchmarking The Multifaceted Trustworthiness of Audio Large Language Models

Kai Li ⋅ Can Shen ⋅ Yile Liu ⋅ Jirui Han ⋅ Kelong zheng ⋅ Xuechao Zou ⋅ Lionel WANG ⋅ Shun Zhang ⋅ Xingjian Du ⋅ Hanjun Luo ⋅ Yingbin Jin ⋅ Xinxin Xing ⋅ Ziyang Ma ⋅ Yue Liu ⋅ YiFan Zhang ⋅ Junfeng Fang ⋅ Kun Wang ⋅ Yibo Yan ⋅ Gelei Deng ⋅ Haoyang LI ⋅ Yiming Li ⋅ Xiaobin Zhuang ⋅ Tianlong Chen ⋅ Qingsong Wen ⋅ Tianwei Zhang ⋅ Yang Liu ⋅ Haibo Hu ⋅ Zhizheng Wu ⋅ Xiaolin Hu ⋅ Ensiong Chng ⋅ Wenyuan Xu ⋅ XiaoFeng Wang ⋅ Wei Dong ⋅ Xinfeng Li

Abstract

The rapid development and widespread adoption of Audio Large Language Models (ALLMs) require a rigorous assessment of their trustworthiness. However, existing evaluation frameworks, primarily designed for text, are not equipped to handle the unique vulnerabilities introduced by audio’s acoustic properties. We find that significant trustworthiness risks in ALLMs arise from non-semantic acoustic cues, such as timbre, accent, and background noise, which can be used to manipulate model behavior. To address this gap, we propose AudioTrust, the first framework for large-scale and systematic evaluation of ALLM trustworthiness concerning these audio-specific risks. AudioTrust spans six key dimensions: fairness, hallucination, safety, privacy, robustness, and authenticition. It is implemented through 26 distinct sub-tasks and a curated dataset of over 4,420 audio samples collected from real-world scenarios (e.g., daily conversations, emergency calls, and voice assistant interactions), purposefully constructed to probe the trustworthiness of ALLMs across multiple dimensions. Our comprehensive evaluation includes 18 distinct experimental configurations and employs human-validated automated pipelines to objectively and scalably quantify model outputs. Experimental results reveal the boundaries and limitations of 14 state-of-the-art (SOTA) open-source and closed-source ALLMs when confronted with diverse high-risk audio scenarios, thereby offering critical insights into the secure and trustworthy deployment of future audio models. Our platform and benchmark are publicly available at https://github.com/JusperLee/AudioTrust.