Current AI systems show hallmarks of human intelligence and creativity that would have been considered far beyond the capabilities of machines only five years ago. The question of how human-like these systems really are is intertwined with ages-old philosophical and scientific debates, as well as the future of our jobs, culture, and lives. Many efforts to assess the presence of human characteristics in AI systems have resorted to analysis of systems' outputs, which, while often illuminating, can also be misleading. In this talk, I’ll discuss the strengths and weaknesses of instead attacking this question by directly measuring similarity between human and machine representations. Unlike approaches that rely only on model outputs, representational approaches can be applied to a broad range of tasks and are not easily fooled by memorization of training data. However, what it means for high dimensional objects such as human and machine representations to be similar remains somewhat unclear, and different definitions of similarity can reach different conclusions. Ultimately, because the problem of comparing human and machine intelligence is so ill-defined, developing a diverse array of techniques that converge on similar conclusions may be our best path forward for reliably characterizing the scope and limits of artificial intelligence.