The goal of this opinion paper is to start (or more accurately, continue) a conversation on a checklist-style artifact for researchers to use when introducing a new dataset. While other checklists exist and are widely used, this paper proposes the inclusion of checklist-style questions to encourage dataset developers (as well as consumers) to consider data quality, diversity, and evaluation, especially as it relates to machine learning model robustness and generalizability