Human language use is exquisitely sensitive to and grounded in perceptual context. I will describe how the "reference game" paradigm allows us to explore these aspects of language use, and to elicit grounded, contextual language for training neural language models. The resulting models predict human language use with high quantitative accuracy. For humans, language is more than a means of communication, it is also the way we distribute the learning problem across generations. I will describe how "concept learning games" allow us to explore this cultural knowledge transmission process and build models that learn from language (somewhat) as people do.