How Learnable Grids Recover Fine Detail in Low Dimensions: A Neural Tangent Kernel Analysis of Multigrid Parametric Encodings

Neural networks that map between low dimensional spaces are ubiquitous incomputer graphics and scientific computing; however, in their naiveimplementation, they are unable to learn high frequency information. We presenta comprehensive analysis comparing the two most common techniques for mitigatingthis spectral bias: Fourier feature encodings (FFE) and multigrid parametricencodings (MPE). FFEs are seen as the standard for low dimensional mappings, butMPEs often outperform them and learn representations with higher resolution andfiner detail. FFE's roots in the Fourier transform, make it susceptible toaliasing if pushed too far, while MPEs, which use a learned grid structure, haveno such limitation. To understand the difference in performance, we use theneural tangent kernel (NTK) to evaluate these encodings through the lens of ananalogous kernel regression. By finding a lower bound on the smallest eigenvalueof the NTK, we prove that MPEs improve a network's performance through thestructure of their grid and not their learnable embedding. This mechanism isfundamentally different from FFEs, which rely solely on their embedding space toimprove performance. Results are empirically validated on a 2D image regressiontask using images taken from 100 synonym sets of ImageNet and 3D implicitsurface regression on objects from the Stanford graphics dataset. Using peaksignal-to-noise ratio (PSNR) and multiscale structural similarity (MS-SSIM) toevaluate how well fine details are learned, we show that the MPE increases theminimum eigenvalue by 8 orders of magnitude over the baseline and 2 orders ofmagnitude over the FFE. The increase in spectrum corresponds to a 15 dB (PSNR) /0.65 (MS-SSIM) increase over baseline and a 12 dB (PSNR) / 0.33 (MS-SSIM) increase over theFFE.