Large Language Models are rapidly emerging as the foundational technology to assist numerous software engineering pains. From code generation to bug fixing to migration and maintenance, they hold the potential to aid every part of the application development lifecycle. However, with great opportunities come great product responsibilities. LLMs need to be adapted to maximize the quality of every application, grounded to the user's context, generate code in a way that respects and uplifts open-source developments upon which they build. I will discuss some of the practical challenges and approaches to real-life LLM adaptation and Code AI product development, using data science as a motivating application.