Efficient Test-Time Adaptation via Decoupled BN Update For Edge Devices

Test-Time Adaptation (TTA) updates a deployed model during online inference to mitigate domain shift. Most TTA methods adapt to the target domain by updating Batch Normalization (BN) layers with unlabeled test data, but often at high memory cost. We observe that BN adaptation consists of two distinct updates, BN statistics and BN affine parameters, which differ in memory footprint and batch size sensitivity. Based on this insight, we propose Decoupled BN Update (DBU), which updates BN statistics with larger batches to capture reliable domain information, while updating affine parameters with small batches to reduce memory overhead. Experiments show that DBU achieves a better performance–memory trade-off, delivering up to 14.5% accuracy improvement in the small-batch setting.