Recent progress of deep generative models in the vision and language domain has stimulated significant interest in more structured data generation such as molecules. However, beyond generating new random molecules, efficient exploration and a comprehensive understanding of the vast chemical space are of great importance to molecular science and applications in drug and materials discovery. In this paper, we propose a new framework, ChemFlow, to traverse the learned chemical space by molecule generative models through learning potential flows in the latent space. Taking a dynamical system perspective, we formulate the problem as learning a vector field that transports the mass of the molecular distribution to the region with desired molecular properties or structure diversity. Under this framework, we unify previous studies on molecule latent space traversal and propose alternative methods incorporating different dynamical priors. We validate the efficacy of ChemFlow on molecule manipulation and single- and multi-objective molecule optimization tasks under both supervised and unsupervised molecular discovery settings.