We introduce the Large Drug Discovery Model (LDDM), a generative framework for target-specific 3D molecule design, leveraging fragment-based masked modeling and large-scale training on the new synthetic dataset of protein-ligand complexes SynthDock. In addition to de novo drug design, LDDM is also able to solve more constrained drug discovery tasks, which allows users to interact with the model during the design process. We furthermore leverage this feature to introduce a new controlled sampling strategy for multi-objective optimization. We benchmarked LDDM across various tasks, including de novo generation, fragment-based drug design, and molecular docking. Finally, we experimentally validated LDDM-designed molecules that bind to the oncogenic target KRAS.