Poster Sat, Apr 25, 2026 • 3:15 PM – 5:45 PM -03

RECODE-H: A Benchmark for Research Code Development with Interactive Human Feedback

Chunyu Miao ⋅ Henry Peng Zou ⋅ Yangning Li ⋅ Yankai Chen ⋅ Yibo Wang ⋅ Fangxin Wang ⋅ Yifan Li ⋅ Wooseong Yang ⋅ Bowei He ⋅ Xinni Zhang ⋅ Dianzhi Yu ⋅ Hanchen Yang ⋅ Hoang Nguyen ⋅ Yue Zhou ⋅ Jie Yang ⋅ Jizhou Guo ⋅ Wenzhe Fan ⋅ Chin-Yuan Yeh ⋅ Panpan Meng ⋅ Liancheng Fang ⋅ Jinhu Qi ⋅ Wei-Chieh Huang ⋅ Zhengyao Gu ⋅ Yuwei Han ⋅ Langzhou He ⋅ Yuyao Yang ⋅ Xue Liu ⋅ Irwin King ⋅ Philip Yu

[ OpenReview]

Abstract

Large language models (LLMs) show the promise in supporting scientific research implementation, yet their ability to generate correct and executable code remains limited. Existing works largely adopt one-shot settings, ignoring the iterative and feedback-driven nature of realistic workflows of scientific research development. To address this gap, we present RECODE-H, a benchmark of 102 tasks from research papers and repositories that evaluates LLMs through multi-turn interactions with human feedback. It includes structured instructions, unit tests, and a five-level feedback hierarchy to reflect realistic researcher–agent collaboration. We further present ReCodeAgent, a framework that integrates feedback into iterative code generation. Experimentswith leading LLMs, including GPT-5, Claude-Sonnet-4, DeepSeek-V3.1, and Gemini 2.5, show substantial performance gains with richer feedback, while also highlighting ongoing challenges in the generation of complex research code. RECODE-H establishes a foundation for developing adaptive, feedback-driven LLM agents in scientific research implementation.

Video

Chat is not available.