MORPHEUS: Meta Test-Time Adaptation via Neural Collapse Geometry

Test-time adaptation (TTA) algorithms make models more accurate on distribution-shifted unlabeled test data. However, the growing diversity of TTA methods has introduced a new challenge: no single TTA strategy consistently dominates across shifts, and several methods suffer catastrophic failures on certain corruption types. This makes it essential to dynamically choose an appropriate TTA method to adapt a given model on a given test data. However, this is challenging as the test data is unlabeled and we cannot run all TTA methods to choose the most accurate one due to prohibitive compute wastage. In this paper, we discover that the entropy of the class distributions and geometric characteristics of the embedding space produced by the source model on unlabeled test data can predict the most accurate TTA method and post-adaptation accuracy without adapting the model. We empirically show that such a selection of the most accurate method can prevent catastrophic failures by choosing an appropriate method. Further, our method can predict the post-adaptation accuracy with an average RMSE of $0.054$ over TTA methods. We believe this will encourage discussions around efficient use of existing TTA methods.