Skip to yearly menu bar Skip to main content


Contributed Talk
in
Workshop: 5th Workshop on practical ML for limited/low resource settings (PML4LRS) @ ICLR 2024

Dˆ2-Sparse: Navigating the low data learning regime with coupled sparse networks

Diganta Misra

[ ]
Sat 11 May 1:35 a.m. PDT — 1:45 a.m. PDT

Abstract:

Research within the realm of deep learning has extensively delved into learning under diverse constraints, with the incorporation of sparsity as a pragmatic constraint playing a pivotal role in enhancing the efficiency of deep learning. This paper introduces a novel approach, termed -Sparse, presenting a dual dynamic sparse learning system tailored for scenarios involving limited data. In contrast to conventional studies that independently investigate sparsity and low-data learning, our research amalgamates these constraints, paving the way for new avenues in sparsity-related investigations. -Sparse outperforms typical iterative pruning methods when applied to standard deep networks, particularly excelling in tasks like image classification within the domain of computer vision. In particular, it achieves a notable 5% improvement in top-1 accuracy for ResNet-34 in the CIFAR-10 classification task, with only 5000 samples compared to iterative pruning methods.

Chat is not available.