Skip to yearly menu bar Skip to main content


(7 events)   Time zone: Conference local time zone  
Show all
Toggle Poster Visibility
Oral
Fri Apr 24 10:30 AM -- 10:40 AM (-03) @ 202 A/B None
Taming Momentum: Rethinking Optimizer States Through Low-Rank Approximation
In Oral Session 3C ML architectures and training I
Zhengbo Wang ⋅ Jian Liang ⋅ Ran He ⋅ Zilei Wang ⋅ Tieniu Tan
[ OpenReview] 
Oral
Fri Apr 24 10:42 AM -- 10:52 AM (-03) @ 202 A/B None
WSM: Decay-Free Learning Rate Schedule via Checkpoint Merging for LLM Pre-training
In Oral Session 3C ML architectures and training I
Changxin Tian ⋅ Jiapeng Wang ⋅ Qian Zhao ⋅ Kunlong Chen ⋅ Jia Liu ⋅ Ziqi Liu ⋅ Jiaxin Mao ⋅ Xin Zhao ⋅ Zhiqiang Zhang ⋅ JUN ZHOU
[ OpenReview] 
Oral
Fri Apr 24 10:54 AM -- 11:04 AM (-03) @ 202 A/B None
Optimal Sparsity of Mixture-of-Experts Language Models for Reasoning Tasks
In Oral Session 3C ML architectures and training I
Taishi Nakamura ⋅ Satoki Ishikawa ⋅ Masaki Kawamura ⋅ Okamoto ⋅ Daisuke Nohara ⋅ Jun Suzuki ⋅ Rio Yokota
[ OpenReview] 
Oral
Fri Apr 24 11:06 AM -- 11:16 AM (-03) @ 202 A/B None
How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining
In Oral Session 3C ML architectures and training I
Kairong Luo ⋅ Zhenbo Sun ⋅ Haodong Wen ⋅ Xinyu Shi ⋅ Jiarui Cui ⋅ Chenyi Dang ⋅ Kaifeng Lyu ⋅ Wenguang Chen
[ OpenReview] 
Oral
Fri Apr 24 11:18 AM -- 11:28 AM (-03) @ 202 A/B None
In-Place Test-Time Training
In Oral Session 3C ML architectures and training I
Guhao Feng ⋅ Shengjie Luo ⋅ Kai Hua ⋅ Ge Zhang ⋅ Wenhao Huang ⋅ Di He ⋅ Tianle Cai
[ OpenReview] 
Oral
Fri Apr 24 11:30 AM -- 11:40 AM (-03) @ 202 A/B None
Softmax Transformers are Turing-Complete
In Oral Session 3C ML architectures and training I
Hongjian Jiang ⋅ Michael Hahn ⋅ Georg Zetzsche ⋅ Anthony W. Lin
[ OpenReview] 
Oral
Fri Apr 24 11:42 AM -- 11:52 AM (-03) @ 202 A/B None
Pre-training under infinite compute
In Oral Session 3C ML architectures and training I
Konwoo Kim ⋅ Suhas Kotha ⋅ Percy Liang ⋅ Tatsunori Hashimoto
[ OpenReview]