Skip to yearly menu bar Skip to main content

Invited Talk
Workshop: Deep Learning for Code

In-IDE Code Generation from Natural Language: Promise and Challenges

Graham Neubig


One major difficulty of programming is turning concepts into code, especially when dealing with the APIs of unfamiliar libraries. Recently, there has been a proliferation of machine learning methods for code generation and retrieval from natural language queries, but these have primarily been evaluated purely based on execution accuracy or overlap of generated code with developer-written code, and the actual effect of these methods on the developer workflow is surprisingly unattested. In this talk, I will describe a user study in which we performed a comprehensive investigation of the promise and challenges of using such technology inside the PyCharm IDE, asking questions such as “does it improve developer productivity or accuracy, how does it affect the developer experience, and what are the remaining gaps and challenges?”

Chat is not available.