Poster Thu, Apr 23, 2026 • 6:30 AM – 9:00 AM PDT Pavilion 3 P3-#1605

AetherCode: Evaluating LLMs’ Ability to Win In Premier Programming Competitions

Zihan Wang ⋅ Jiaze Chen ⋅ Zhicheng Liu ⋅ Haojie Pan ⋅ Markus Mak ⋅ Yidi Du ⋅ Geonsik Moon ⋅ Aaron Tua ⋅ Kunshuo Peng ⋅ Jiayi Lu ⋅ Boqian Zou ⋅ Chenyang Ran ⋅ GuangTian ⋅ Shoutai Zhu ⋅ Duan Yeheng ⋅ Zhenghui Kang ⋅ Zhenxing Lin ⋅ Lishangshu ⋅ Qiang Luo ⋅ Qingshen Long ⋅ Zhiyong Chen ⋅ Yihan Xiao ⋅ Yurong Wu ⋅ Daoguang Zan ⋅ Mingxuan Wang ⋅ Ming Ding

Abstract

Competitive programming has emerged as a critical benchmark for evaluating the reasoning and coding capabilities of Large Language Models (LLMs). Despite impressive progress on existing benchmarks, we argue that current evaluations overstate model proficiency, masking a substantial gap between LLMs and elite human programmers. This gap arises from two key limitations: insufficient difficulty and scope of benchmark problems, and evaluation bias from low-quality test cases. To address these shortcomings, we present AetherCode, a new benchmark that draws problems from premier programming competitions such as IOI and ICPC, offering broader coverage and higher difficulty. AetherCode further incorporates comprehensive, expert-validated test suites built through a hybrid of automated generation and human curation, ensuring rigorous and reliable assessment. By combining challenging problem design with robust evaluation, AetherCode provides a more faithful measure of LLM capabilities and sets a new standard for future research in code reasoning.