(16:15–16:30) Constructing Machine-Precision Neural Networks with Quasi-Interpolants

Neural networks struggle to train to machine precision for simple interpolation tasks, limiting their use in scientific computing pipelines. We address this by providing the first explicit MLP construction that provably achieves machine-precision interpolation with $\log(1/\varepsilon)$ parameter scaling --- matching classical polynomial methods --- while remaining implementable in floating-point arithmetic. Our construction, based on quasi-interpolation theory, reveals a critical dimensionless bandwidth parameter $\lambda$ that controls an aliasing vs numerical stability tradeoff; setting the optimal $\lambda$ implies weight magnitudes must grow with width. Using this framework to analyze trained MLPs, we find that they do not maintain the required weight scaling and exhibit rank saturation. Our results provide a principled approach for understanding how optimization, not expressivity, underlies precision failures in scientific machine learning.