Constructing Machine-Precision Neural Networks with Quasi-Interpolants

Neural networks struggle to train to machine precision for simple interpolation tasks, limiting their use in scientific computing pipelines. We address this by providing the first explicit MLP construction that provably achieves machine-precision interpolation with $\log(1/\varepsilon)$ parameter scaling --- matching classical polynomial methods --- while remaining implementable in floating-point arithmetic. Our construction, based on quasi-interpolation theory, reveals a critical bandwidth parameter $\lambda$ that controls a tradeoff between aliasing error and conditioning; setting the optimal $\lambda$ implies weight magnitudes must grow with width. Using this framework to analyze trained MLPs, we find that they do not maintain the required weight scaling and exhibit rank saturation. Our results provide a principled framework for understanding why optimization, not expressivity, underlies precision failures in scientific machine learning.