CA-BED: Conversation-Aware Bayesian Experimental Design

Large Language Models (LLMs) excel at static reasoning tasks, yet their reliability diminishes in interactive scenarios where information must be actively acquired through questioning. A key challenge lies in asking questions that minimize uncertainty while simultaneously internalizing responses that may be ambiguous or partially specified. To address this, we propose Conversation-Aware Bayesian Experimental Design (CA-BED), an inference-time probabilistic dialogue planning framework that integrates Bayesian Experimental Design with LLM-based likelihood estimation to optimize question selection over multiple conversational turns. CA-BED models uncertainty continuously, anticipates potential answers, and propagates expected information gain through a simulated conversation tree, enabling more efficient and robust information acquisition. Across entity-deduction benchmarks, CA-BED yields an average 25% improvement in success rates over direct prompting, with comparable gains relative to alternative information-seeking methods. It further achieves a 10.25% average reduction in conversation length, indicating greater efficiency in information gathering. These results highlight CA-BED’s effectiveness as a principled framework for reliable interactive reasoning in real-world settings.