Evidence Slopes and Effective Dimension in Singular Linear Models

Bayesian model selection often uses Laplace’s approximation or BIC, which assume the complexity penalty in the log evidence is $(d/2)\log n$, where $d$ is the parameter dimension. Singular learning theory replaces $d/2$ with the real log canonical threshold (RLCT) $\lambda$, which can be strictly smaller in overparameterized low-rank models. We study linear–Gaussian rank and subspace/dictionary models where the marginal likelihood is available in closed form and $\lambda$ is analytically identifiable. We prove and empirically verify that Laplace/BIC incur a leading bias $((d/2)-\lambda)\log n$; in rank-$r$ regression, $\lambda=r/2$. In a dictionary model, the leading evidence term is invariant under overcomplete reparameterizations with the same span, while BIC is not. These results provide a clean finite-sample benchmark that makes “Laplace failure” in singular models explicit and motivates slope-based diagnostics for effective dimension.