Addax: Memory-Efficient Fine-Tuning of Language Models with a Combination of Forward-Backward and Forward-Only Passes
Zeman Li, Xinwei Zhang, Meisam Razaviyayn

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
Jiawei Zhao, Zhenyu Zhang, Beidi Chen, Zhangyang Wang, Anima Anandkumar, Yuandong Tian

Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models
Keisuke Kamahori, Yile Gu, Kan Zhu, Baris Kasikci

Autoregressive activity prediction for low-data drug discovery
Johannes Schimunek, Lukas Friedrich, Daniel Kuhn, Günter Klambauer

Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity
Lu Yin, You Wu, Zhenyu Zhang, Cheng-Yu Hsieh, Yaqing Wang, Yiling Jia, Gen Li, AJAY KUMAR JAISWAL, Mykola Pechenizkiy, Yi Liang, Michael Bendersky, Zhangyang Wang, Shiwei Liu

SparQ Attention: Bandwidth-Efficient LLM Inference
Luka Ribar, Ivan Chelombiev, Luke Hudlass-Galley, Charlie Blake, Carlo Luschi, Douglas Orr

How to Parameterize Asymmetric Quantization Ranges for Quantization-Aware Training
Jaeseong You, Minseop Park, Markus Nagel, Kyunggeun Lee, Seokjun An, Chirag S Patel

SPI-GAN: Denoising Diffusion GANs with Straight-Path Interpolations Jinsung Jeon, Noseong Park

On the Surprising Efficacy of Distillation as an Alternative to Pre-Training Small Models
Sean Farhat, Deming Chen

SSM Meets Video Diffusion Models: Efficient Video Generation with Structured State Spaces
Yuta Oshima, Shohei Taniguchi, Masahiro Suzuki, Yutaka Matsuo

Multi-model evaluation with labeled & unlabeled data
Divya M Shanmugam, Shuvom Sadhuka, Manish Raghavan, John Guttag, Bonnie Berger, Emma Pierson

Defer-and-Fusion: Optimal Predictors that Incorporate Human Decisions
Mohammad-Amin Charusaie, Amirmehdi Jafari Fesharaki, Samira Samadi

Selective Prediction for Semantic Segmentation under Distribution Shift Bruno Laboissiere Camargos Borges, Bruno Machado Pacheco, Danilo Silva