Quantum Machine Learning (QML) is a nascent field of technology that is yet to be fully explored. While previous QML implementations have demonstrated performance efficiency gains over classical benchmarks, it has not been studied in detail whether shallow unentangled quantum circuits can provide the same benefits to reinforcement learning algorithms. Towards this goal, we present a shallow Deep Q-Network (DQN) hybrid quantum-classical Variational Quantum Circuit (VQC) model in the Cartpole-v0 environment that provides an increase in training stability and average reward for any given training run with a simpler unentangled quantum circuit than what is proposed in prior literature.