Poster Thu, Apr 23, 2026 • 6:30 AM – 9:00 AM PDT Pavilion 3 P3-#1512

DiscoX: Benchmarking Discourse-Level Translation in Expert Domains

Xiying ZHAO ⋅ Zhoufutu Wen ⋅ Zhixuan Chen ⋅ Jingzhe Ding ⋅ Jianpeng Jiao ⋅ Shuai Li ⋅ Xi Li ⋅ Danni Liang ⋅ Shengda Long ⋅ Qianqian Liu ⋅ Xianbo Wu ⋅ Hongwan Gao ⋅ Xiang Gao ⋅ LIANG HU ⋅ Jiashuo Liu ⋅ Liumengyun ⋅ Weiran Shi ⋅ Chenghao Yang ⋅ Qianyu Yang ⋅ Xuanliang Zhang ⋅ Ge Zhang ⋅ Wenhao Huang

Abstract

The evaluation of discourse-level translation in expert domains remains inadequate, despite its centrality to knowledge dissemination and cross-lingual scholarly communication. While these translations demand discourse-level coherence and strict terminological precision, current evaluation methods predominantly focus on segment-level accuracy and fluency. To address this limitation, we introduce DiscoX, a new benchmark for discourse-level and expert-level Chinese-English translation. It comprises 200 professionally-curated texts from 7 domains, with an average length exceeding 1700 tokens. To evaluate performance on DiscoX, we also develop Metric-S, a reference-free system that provides fine-grained automatic assessments across accuracy, fluency, and appropriateness. Metric-S demonstrates strong consistency with human judgments, significantly outperforming existing metrics. Our experiments reveal a remarkable performance gap: even the most advanced LLMs still trail human experts on these tasks. This finding validates the difficulty of DiscoX and underscores the challenges that remain in achieving professional-grade machine translation. The proposed benchmark and evaluation system provide a robust framework for more rigorous evaluation, facilitating future advancements in LLM-based translation. Our data and code are available at https://github.com/ByteDance-Seed/DiscoX.