Maximum Entropy under Carre du Champ Constraints

Recent works have shown generative models' striking capabilities to memorize training data, and encode the underlying lower-dimensional data manifold $M$. However, while this phenomenon has been studied from different perspectives, current training objectives do not explicitly \textit{control} the geometric complexity of memorization. In this work, we propose a constrained maximum-entropy (MaxEnt) principle for learning a generative model density $q$ that (i) fits the data, (ii) discourages tangential ``memorization'' (training-point attractors along the manifold) through a data-driven Carr\'{e}-du-Champ (CDC) tangential Fisher constraint defined on the projected marginal of $q$ onto $M$, and (iii) enforces bounded normal thickness so the ambient entropy objective is well-posed. Leveraging this, we prove KL-to-uniform control and explicit anti-concentration bounds near training points, and lift them to ambient space via thickness control. We also provide scalable kNN/local-PCA estimators for the data-driven geometric terms and demonstrate numerical and simulation results. More generally, to our knowledge, this is the first work proposing a principled variational principle that jointly controls entropy and geometric complexity through a data-dependent metric.