Skip to yearly menu bar Skip to main content


Poster

Divergence of Neural Tangent Kernel in Classification Problems

Zixiong Yu · Songtao Tian · Guhan Chen

Hall 3 + Hall 2B #314
[ ]
Sat 26 Apr midnight PDT — 2:30 a.m. PDT

Abstract:

This paper primarily investigates the convergence of the Neural Tangent Kernel (NTK) in classification problems. This study firstly show the strictly positive definiteness of NTK of multi-layer fully connected neural networks and residual neural networks. Then, through a contradiction argument, it indicates that, during training with the cross-entropy loss function, the neural network parameters diverge due to the strictly positive definiteness of the NTK. Consequently, the empirical NTK does not consistently converge but instead diverges as time approaches infinity. This finding implies that NTK theory is not applicable in this context, highlighting significant theoretical implications for the study of neural networks in classification problems. These results can also be easily generalized to other network structures, provided that the NTK is strictly positive definite.

Live content is unavailable. Log in and register to view live content