Stop Automating Peer Review Without Rigorous Evaluation

As AI systems increasingly generate scientific knowledge, the human ability to critically evaluate research becomes more important, not less. Yet large language models offer a tempting solution to address the peer review crisis, risking the automation of the very skills scientists will need most. This position paper argues that today's AI systems should not be used to produce paper reviews. We ground this position in an empirical comparison of human- versus AI-generated ICLR 2026 reviews and an evaluation of the effect of automated paper rewriting on different AI reviewers. We identify two critical issues: 1) AI reviewers exhibit a hivemind effect of excessive agreement within and across papers that reduces perspective diversity. 2) AI review scores are trivially gameable through paper laundering: prompting an LLM to rewrite a paper significantly increases scores from AI reviewers through stylistic changes rather than scientific improvements. However, non-gameability and review diversity are necessary but not sufficient conditions for automation. We argue that addressing the peer review crisis requires a science of peer review automation that keeps human scientific judgment at the center of the process---especially as we enter an era where that judgment will be needed most.