Skip to yearly menu bar Skip to main content


Workshop Sun, Apr 26, 2026 • 5:00 AM – 1:00 PM PDT 101 B

Scientific Methods for Understanding Deep Learning (Sci4DL)

Zahra Kadkhodaie ⋅ Florentin Guth ⋅ Sanae Lotfi ⋅ Davis Brown ⋅ Antonio Sclocchi ⋅ Sharvaree Vadgama ⋅ James Simon ⋅ Eero Simoncelli
Project Page

Abstract

Video

Chat is not available.

Schedule

Timezone: America/Los_Angeles
6:30 AM
Coffee Break
8:30 AM
Lunch Break
10:30 AM
Coffee Break
11:00 AM
Panel Discussion - Jeremy Cohen, Julia Kempe, David Bau, Richard Baraniuk, Matthieu Wyart
Jeremy Cohen ⋅ Julia Kempe ⋅ David Bau ⋅ Richard Baraniuk ⋅ Matthieu Wyart
12:15 PM
Poster Session 2
Expert-Data Alignment Governs Generation Quality in Decentralized Diffusion Models
Marcos Villagra ⋅ Bidhan Roy ⋅ Raihan Seraj ⋅ Zhiying Jiang
Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models
Julianna Piskorz ⋅ Cristina Pinneri ⋅ Alvaro Correia ⋅ Motasem Alfarra ⋅ Risheek Garrepalli ⋅ Christos Louizos
Generating output diversity from prompt re-tokenization
Kanishk Jain ⋅ Matthew Day ⋅ Tankut Can
The Feature-Space Alignment Hypothesis for Neural Network Sparsity
Linghao Kong ⋅ Micah Adler ⋅ Nir Shavit
Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models
Jakub Binkowski ⋅ Kamil Adamczewski ⋅ Tomasz Kajdanowicz
Skip To The Good Part: Representation Structure & Inference-Time Layer Skipping in Diffusion vs Autoregressive LLM
Raghavv Goel ⋅ Risheek Garrepalli ⋅ Sudhanshu Agrawal ⋅ Christopher Lott ⋅ Fatih Porikli ⋅ Mingu Lee
Configuration-to-Performance Scaling Law with Neural Ansatz
Huaqing Zhang ⋅ Kaiyue Wen ⋅ Tengyu Ma
In-Context Benign Overfitting: A Feature-Selection Model in In-Context Linear Regression
Puneesh Deora ⋅ Bhavya Vasudeva ⋅ Christos Thrampoulidis
Neural Multivariate Regression with Multi-Task Learning and Target Preprocessing
George Andriopoulos ⋅ Soyuj Jung Basnet ⋅ Juan Guevara ⋅ Li Guo ⋅ Bimarsha Adhikari ⋅ Keith Ross
On the "Induction Bias" in Sequence Models
MohammadReza Ebrahimi ⋅ Michaël Defferrard ⋅ Sunny Panchal ⋅ Roland Memisevic
Toy Models of Combinatorial Interpretability
Nir Shavit ⋅ Dan Alistarh ⋅ Micah Adler
Weight Decay Improves Language Model Plasticity
Tessa Han ⋅ Sebastian Bordt ⋅ Hanlin Zhang ⋅ Sham Kakade
Optimal learning rate scaling depends on data in deep scalar linear networks
Yedi Zhang ⋅ Peter Latham ⋅ Leena Chennuru Vankadara ⋅ Andrew Saxe
Entropy-Lens: Uncovering Decision Strategies in LLMs
Christopher Irwin ⋅ Francesco Caso ⋅ Riccardo Ali ⋅ Pietro Lio
Leveraging Low-Rank Structure for Effective Weight-Sharing in Language Models
Mark Muchane ⋅ George Sokolik ⋅ Sanae Lotfi ⋅ Micah Goldblum
Network of Theseus (Like the ship)
Vighnesh Subramaniam ⋅ Colin Conwell ⋅ Boris Katz ⋅ Andrei Barbu ⋅ Brian Cheung
On the Simplicity-Similarity Tradeoff of LoRA and Full Fine-Tuning
Jérôme Emery ⋅ Darshan Patil ⋅ François Leduc-Primeau ⋅ Sarath Chandar ⋅ Ekaterina Lobacheva
Revealing Task-Dependent Layer Relevance via Attentive Multi-Layer Fusion
Marco Morik ⋅ Laure Julie Ciernik ⋅ Lukas Thede ⋅ Luca Eyring ⋅ Shinichi Nakajima ⋅ Zeynep Akata ⋅ Lukas Muttenthaler
Thermodynamics of Reinforcement Learning Curricula
Jacob Adamczyk ⋅ Juan Rojas ⋅ Rahul Kulkarni
Single-Head Attention in High Dimensions: A Theory of Generalization, Weights Spectra, and Scaling Laws
Fabrizio Boncoraglio ⋅ Vittorio Erba ⋅ Emanuele Troiani ⋅ Yizhou Xu ⋅ Florent Krzakala ⋅ Lenka Zdeborova
Shared Gradient Discovery and Superposition: Learning Dynamics of Generalization in LLMs
Andrei Mircea ⋅ Ildus Sadrtdinov ⋅ Irina Rish ⋅ Ekaterina Lobacheva
Special solutions with small volume exist
Tausifa Jan Saleem ⋅ Ramanjit Ahuja ⋅ Surendra Prasad ⋅ Brejesh Lall
TrasMuon: Trust-Region Adaptive Scaling for Orthogonalized Momentum Optimizers
Peng Cheng ⋅ Jiucheng Zang ⋅ Qingnan Li ⋅ Liheng Ma ⋅ Yufei CUI ⋅ Yingxue Zhang ⋅ Boxing Chen ⋅ Ming Jian ⋅ Wen Tong
Ablate and Rescue: A Causal Analysis of Residual Stream Hyper-Connections
William Peng ⋅ Josheev Rai ⋅ Kevin Tseng ⋅ Siwei Wang ⋅ Sean Wu
Learning When to Be Sparse: Adaptive Activations via Two-Parameter Entropy
Roman Rudamenko ⋅ Dmitry Abulkhanov ⋅ Konstantin Semenov ⋅ Michael Diskin ⋅ Alexander Savchenko
Do Depth-Grown Models Overcome the Curse of Depth? An In-Depth Analysis
Ferdinand Kapl ⋅ Emmanouil Angelis ⋅ Tobias Höppe ⋅ Kaitlin Maile ⋅ Johannes von Oswald ⋅ Nino Scherrer ⋅ Stefan Bauer
Pretraining with Masked Backstories in a Toy World
Sultan Daniels ⋅ Dylan Davis ⋅ Gireeja Ranade ⋅ Anant Sahai
Does LLM Pre-Training Typically Occur at the Edge of Stability?
Yuhang Cai ⋅ Haofeng Huang ⋅ Haodong Wen ⋅ Deyi Liu ⋅ Yiyuan Ma ⋅ Kaifeng Lyu
From Growing to Looping: A Unified View of Iterative Computation in LLMs
Ferdinand Kapl ⋅ Emmanouil Angelis ⋅ Kaitlin Maile ⋅ Johannes von Oswald ⋅ Stefan Bauer
Model Evolution Under Zeroth-Order Optimization: A Neural Tangent Kernel Perspective
Chen Zhang ⋅ Yuxin Cheng ⋅ Chenchen Ding ⋅ Shuqi Wang ⋅ Jingreng Lei ⋅ Runsheng Yu ⋅ Yik-Chung WU ⋅ Ngai Wong
Spherical Cautious Optimizers
Jinghui Yuan ⋅ Feiping Nie
Process-then-Retrieve: A Mechanistic Study of Cross-Modal Alignment in Vision-Language Models
Arpita Shanbhag ⋅ Julia Tran ⋅ Dhruv Mandala ⋅ Ayda Sultan
Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
Bhavya Vasudeva ⋅ Puneesh Deora ⋅ Alberto Bietti ⋅ Vatsal Sharan ⋅ Christos Thrampoulidis
Less Data, Faster Training: sampling bias from small dataset can speed up training
Jingwen Liu ⋅ Ezra Edelman ⋅ Surbhi Goel ⋅ Bingbin Liu
Optimal scaling laws in learning hierarchical multi-index models
Leonardo Defilippis ⋅ Florent Krzakala ⋅ Bruno Loureiro ⋅ Antoine Maillard
Scaling-Law Analysis of SignSGD: From Feature-Space Linear Regression to LLM Pre-training
Zilin Wang ⋅ Binghui Li ⋅ Lean Wang ⋅ Jianan Wang ⋅ Jinbo Wang ⋅ Lei Wu
Unified Perspectives on Balancedness and Parameter-norm Evolution in Neural Nets
Jasraj Singh ⋅ Enea Monzio Compagnoni ⋅ Antonio Orvieto
Zeroth-Order Optimization at the Edge of Stability
Minhak Song ⋅ Liang Zhang ⋅ Bingcong Li ⋅ Niao He ⋅ Michael Muehlebach ⋅ Sewoong Oh
Dropout and the Outliers: Could Transformers Overcome Their Single Points of Failure?
Nour Hezbri ⋅ Gilles Bareilles ⋅ El-Mahdi El-Mhamdi
Simple LLM Baselines are Competitive for Model Diffing
Elias Kempf ⋅ Simon Schrodi ⋅ Bartosz Cywiński ⋅ Thomas Brox ⋅ Neel Nanda ⋅ Arthur Conmy
Training for Compositional Sensitivity Reduces Dense Retrieval Generalization
Radoslav Ralev ⋅ Aditeya Baral ⋅ Iliya Zhechev ⋅ Jen Agarwal ⋅ Srijith Rajamohan
Instruction Following by Principled Attention Boosting of Large Language Models
Vitoria Guardieiro ⋅ Avishree Khare ⋅ Adam Stein ⋅ Eric Wong
Understanding Learning Dynamics of Zeroth-Order Optimization
Zhe Li ⋅ Bicheng Ying ⋅ Zidong Liu ⋅ Haibo Yang
The Role of Data in Model Merging
Gaurav Iyer ⋅ Ekaterina Lobacheva
Genomic Next-Token Predictors are In-Context Learners
Nathan Breslow ⋅ Aayush Mishra ⋅ Michael Schatz ⋅ Anqi Liu ⋅ Mahler Revsine ⋅ Daniel Khashabi
Analysing the Linearity of Linguistic Relations in Language Model Embedding Spaces
Vasudevan Nedumpozhimana ⋅ Fathima Thekkekara ⋅ John Kelleher
Steered LLM Activations are Non-Surjective
Aayush Mishra ⋅ Daniel Khashabi ⋅ Anqi Liu
Understanding Scaling Laws With Token-Level Analysis
Arkil Patel ⋅ Marius Mosbach ⋅ Siva Reddy ⋅ Dzmitry Bahdanau
Endogenous Resistance to Activation Steering in Language Models
Alex McKenzie ⋅ Keenan Pepper ⋅ Stijn Servaes ⋅ Martin Leitgab ⋅ Murat Cubuktepe ⋅ Michael Vaiana ⋅ Diogo de Lucena ⋅ Judd Rosenblatt ⋅ Michael Graziano
When does Observational Data Teach Latent Dynamics? Understanding Control Misalignment with Synthetic Tasks
Kento Nishi ⋅ Raphael Tang ⋅ Karun Kumar ⋅ Core Francisco Park ⋅ Hidenori Tanaka
Geometric Properties of Neural Multivariate Regression: An Empirical Study
George Andriopoulos ⋅ Zixuan Dong ⋅ Bimarsha Adhikari ⋅ Keith Ross
STRIDE: Training Data Attribution Can Be Estimated In Activation Space
Abir Harrasse ⋅ Rishit Dagli ⋅ Amir Abdullah ⋅ Zhijing Jin
The Offline-Frontier Shift: Diagnosing Distributional Limits in Generative Multi-Objective Optimization
Stephanie Holly ⋅ Alexandru-Ciprian Zavoianu ⋅ Siegfried Silber ⋅ Sepp Hochreiter ⋅ Werner Zellinger
Deriving Hyperparameter Scaling Laws via Modern Optimization Theory
Egor Shulgin ⋅ Dimitri von Rütte ⋅ Tianyue Zhang ⋅ Niccolò Ajroldi ⋅ Bernhard Schölkopf ⋅ Antonio Orvieto
What Flow-Matching Brings to TD Learning?
Bhavya Agrawalla ⋅ Michal Nauman ⋅ Aviral Kumar
Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting
Ishaan Watts ⋅ Catherine Li ⋅ Sachin Goyal ⋅ Jacob Springer ⋅ Aditi Raghunathan
When to restart? Exploring escalating restarts on convergence
Ayush Kumar Varshney ⋅ Sarunas Girdzijauskas ⋅ Konstantinos Vandikas ⋅ Aneta Feljan