Flipping the Dialogue: Training and Evaluating User Language Models

Conversations with LMs involve two participants: a human user leading the conversation, and an LM assistant responding to the user's request. To satisfy this specific role, LMs are post-trained to be helpful assistants - optimized to produce exhaustive and well-structured responses, often free of ambiguity. User utterances, on the other hand, are rarely perfected, with each user phrasing requests in unique and indirect ways, making the least effort at each turn and refining on the fly. To better understand LM performance in a realistic setting, prior work has proposed to simulate users in multi-turn conversations, often prompting an assistant model to play the role of the user. However, we show in this work that assistant LMs make for poor user simulators, with surprising evidence that better assistants yield worse simulators. Instead, we introduce purpose-built User Language Models (User LMs) - models post-trained to simulate human users in multi-turn conversations. Through various evaluations, we show how User LMs align better with human behavior and achieve better simulation robustness than existing simulation methods. When leveraging User LMs to simulate coding and math conversations, the performance of a strong assistant (GPT-4o) drops from 74.6% to 57.4%, confirming that more realistic simulation environments lead to assistant struggles as they fail to cope with the nuances of users in multi-turn setups.