Recent advances in diffusion models have enabled increasing capabilities for inverse materials design. The key capability is achieving tailored design towards desired property profiles, with wide applications for climate change, semiconductor design, and catalysis. In this work, we present MatInvent, a reinforcement learning (RL) framework tailored to optimize diffusion models for goal-directed crystal generation. By formulating equivariant denoising as a multi-step decision-making problem, MatInvent leverages policy optimization with reward-weighted KL regularization, including experience replay and diversity filters to enhance sample efficiency and diversity. Experimental results demonstrate that MatInvent outperforms existing baselines, offering an effective strategy for crystal generation with single or multiple property optimization.