Automated Machine Learning Research via Agentic Exploration with Human Oversight

This paper proposes a scalable approach to automated incremental machine learning research that integrates agentic exploration (autonomous hypothesis generation) with human oversight (human-verified evaluation) to ensure accountability.Our framework systematically generates novel neural network components, validates their feasibility, evaluates performance against established baselines, and conducts an autogenerated meta-analysis to uncover common patterns of success across the proposed components, and explain them in natural language. We also investigate training a reward model capable of identifying successful patterns within the abstract embedding space, which can be used to prioritize more promising hypotheses pre-evaluation, thus improving hypothesis generation efficiency.