Genetic perturbation experiments play a crucial role in discovering the mechanisms behind diseases and informing drug development. These experiments aim to find a small subset out of many possible genes that yield a particular phenotype (e.g. cell growth) upon perturbation. However, the costs involved in each experiment limits the number of perturbations that can be tested. Here, we develop BioDiscoveryAgent, an AI agent that can strategically design genetic perturbation experiments to enhance the detection of perturbations that induce desired phenotypes. Our AI agent is based on large language models, which have rich biological knowledge, and generate explainable rationales while selecting genes to perturb. BioDiscoveryAgent attains an average of 23% improvement compared to existing Bayesian optimization baselines in detecting desired phenotypes across five datasets. This includes one dataset that is unpublished and therefore guaranteed to not appear in the language model's training data. Additionally, BioDiscoveryAgent is uniquely able to predict gene combinations to perturb, a task so far not explored in this setting. Overall, our approach represents a simple new paradigm in computational design of biological experiments, aimed at augmenting scientists' capabilities and accelerating scientific discovery.