Silicon Reviewers: Simulating Expert Panels to Test Funding Mechanisms

Inter-reviewer agreement in peer review hovers around $\kappa = 0.17$, barely above chance; a quarter of funding decisions would reverse with different reviewers. Yet we lack methods for testing whether proposed reforms improve outcomes, because experiments on live funding are expensive and ethically fraught. We propose the \textit{grant simulation machine}: AI agents trained on historical evaluation data to replicate expert panel behavior, validated against real decisions, then used to test counterfactual funding mechanisms in silico. Drawing on AI-based human simulation (85\% accuracy modeling individual behavior) and a partnership providing complete evaluation-to-outcome data, we outline a framework for comparing golden tickets, partial lotteries, and alternative panel compositions. The approach faces real limitations: AI may not capture emergent group dynamics, and simulated outcomes cannot substitute for actual scientific productivity. But it offers something currently lacking: a low-stakes laboratory for institutional innovation.