The intersection of machine learning and dynamical systems has generated considerable interest recently. Neural Ordinary Differential Equations (NODEs) represent a rich overlap between these fields. In this paper, we develop a continuous-time neural network approach based on Delay Differential Equations (DDEs). Our model uses the adjoint sensitivity method to learn the model parameters and delay directly from data. Our approach builds upon recent developments in NODEs and extends earlier neural DDE models, which assume the delay is known a priori. We rigorously justify our adjoint method and use numerical experiments to demonstrate our algorithm's ability to learn delays and parameters from data. Since the delay is rarely known \emph{a. priori}, our approach advances system identification of DDEs from real-world data.