Workshop on Neural Architecture Search

Arber Zela, Aaron Klein, Frank Hutter, Liam Li, Jan Hendrik Metzen, Jovita Lukasik


Neural Architecture Search (NAS) is an exciting new field of study that is taking representation learning to the next level by allowing us to learn the architectures in a data-driven way that then enables efficient learning of representations. While representation learning removed the need of manual feature engineering, it shifted the manual task to the manual selection of architectures; as a natural next step, NAS replaces this manual architecture selection step, allowing us true end-to-end learning of the architecture, the features, and the final classifier using the features expressed as instantiations of the architecture.

Since the first workshop on NAS at ICLR 2020, there have been many new developments in NAS. Firstly, there has been a large increase in standardized tabular benchmarks and more researchers releasing source code, leading to more rigorous empirical NAS research and also allowing research groups without access to industry-scale compute resources to run thorough experimental evaluations. Secondly, there are now several works aiming for standardized and modularized open-source libraries that allow for both clean evaluations of different approaches without confounding factors and for mixing and matching components of different NAS methods. Finally, by now there are also several applications of NAS beyond its original narrow focus on object recognition, to fields like semantic segmentation, speech recognition, and natural language processing.

In this workshop, we want to push NAS to the next level and aim to address questions (see proposal) which are of particular relevance to the NAS community. In terms of prospective participants, our main targets are machine learning researchers interested in understanding and improving current NAS methods, but ML researchers planning to apply existing NAS methods to novel domains are also amongst the target community.

Chat is not available.

Timezone: »


Fri 6:00 a.m. - 6:11 a.m.
Welcome + Live Introduction of Fabio Carlucci (Live Introduction)
Fri 6:11 a.m. - 6:46 a.m.
Invited Talk Fabio Carlucci (Invited Talk 1)   
Fabio Carlucci
Fri 6:46 a.m. - 7:00 a.m.
Q&A Fabio Carlucci (Q&A Session)
Fri 7:00 a.m. - 7:02 a.m.
Poster Spotlight "HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Search" (Poster Spotlight)   
Niv Nayman
Fri 7:02 a.m. - 7:04 a.m.
Poster Spotlight "How Powerful are Performance Predictors in Neural Architecture Search?" (Poster Spotlight)   
Colin White, Arber Zela
Fri 7:04 a.m. - 7:06 a.m.
Poster Spotlight "Overfitting in Bayesian Optimization: an empirical study and early-stopping solution" (Poster Spotlight)   
Huibin Shen, Anastasia Makarova
Fri 7:06 a.m. - 7:08 a.m.
Poster Spotlight "Width transfer: on the (in)variance of width optimization" (Poster Spotlight)   
Ting-Wu Chin
Fri 7:08 a.m. - 7:10 a.m.
Poster Spotlight "One-Shot Neural Architecture Search Via Compressive Sensing" (Poster Spotlight)   
Minsu Cho, Mohammadreza Soltani, Chinmay Hegde
Fri 7:10 a.m. - 7:12 a.m.
Poster Spotlight "Measuring Uncertainty through Bayesian Learning of Deep Neural Network Structure" (Poster Spotlight)   
Zhijie Deng
Fri 7:12 a.m. - 7:14 a.m.
Poster Spotlight "On Adversarial Robustness: A Neural Architecture Search perspective" (Poster Spotlight)   
Chaitanya Devaguptapu
Fri 7:14 a.m. - 7:16 a.m.
Poster Spotlight "MONCAE: Multi-Objective Neuroevolution of Convolutional Autoencoders" (Poster Spotlight)   
Daniel Dimanov, Emili Balaguer-Ballester
Fri 7:16 a.m. - 7:18 a.m.
Poster Spotlight "AlphaNet: Improved Training of Supernet with Alpha-Divergence" (Poster Spotlight)   
Dilin Wang
Fri 7:18 a.m. - 7:20 a.m.
Poster Spotlight "Flexible Multi-task Networks by Learning Parameter Allocation" (Poster Spotlight)   
Krzysztof Maziarz, Efi Kokiopoulou
Fri 7:20 a.m. - 8:20 a.m.
 link »

Please join us in GatherTown for our poster session.

Fri 8:20 a.m. - 8:30 a.m.
Coffee Break (Break)
Fri 8:30 a.m. - 8:31 a.m.
Live introduction for Xuanyi Dong (Live intro)
Fri 8:31 a.m. - 9:06 a.m.
Invited Talk Xuanyi Dong (Invited Talk)   
Xuanyi Dong
Fri 9:06 a.m. - 9:20 a.m.
Q&A Xuanyi Dong (Q&A Session)
Fri 9:20 a.m. - 9:21 a.m.
Live Intro Contributor Talks (Live intro)
Fri 9:21 a.m. - 9:36 a.m.
Contributor Talk 1 (Invited Talk)   
Niv Nayman
Fri 9:36 a.m. - 9:40 a.m.
Q&A Contributor Talk 1 (Q&A Session)
Fri 9:40 a.m. - 9:55 a.m.
Contributor Talk 2 (Invited Talk)   
Colin White, Arber Zela
Fri 9:55 a.m. - 10:00 a.m.
Q&A Contributor Talk 2 (Q&A Session)
Fri 10:00 a.m. - 11:00 a.m.
Break (break)
Fri 11:00 a.m. - 11:01 a.m.
Live introduction for Margret Keuper (Live intro)
Fri 11:01 a.m. - 11:36 a.m.
Invited Talk Margret Keuper (Invited Talk)   
Margret Keuper
Fri 11:36 a.m. - 11:50 a.m.
Q&A Margret Keuper (Q&A Session)
Fri 11:50 a.m. - 11:52 a.m.
Poster Spotlight "Making Differentiable Architecture Search less local" (Poster Spotlight)   
Erik Bodin, Zhenwen Dai
Fri 11:52 a.m. - 11:54 a.m.
Poster Spotlight "A multi-objective perspective on tuning hardware and hyperparameters" (Poster Spotlight)   
David Salinas Salinas
Fri 11:54 a.m. - 11:56 a.m.
Poster Spotlight "Simulation-based Scoring for Model-based Asynchronous Hyperparameter and Neural Architecture Search" (Poster Spotlight)   
Matthias Seeger
Fri 11:56 a.m. - 11:58 a.m.
Poster Spotlight "AutoHAS: Efficient Hyperparameter and Architecture Search" (Poster Spotlight)   
Xuanyi Dong
Fri 11:58 a.m. - 12:00 p.m.
Poster Spotlight "Tensorizing Neural Architecture Search in the Supernet" (Poster Spotlight)   
Hansi Yang, James Kwok, Quanming Yao
Fri 12:00 p.m. - 12:02 p.m.
Poster Spotlight "Recovering Quantitative Models of Human Information Processing with Differentiable Architecture Search" (Poster Spotlight)   
Sebastian Musslick
Fri 12:02 p.m. - 12:04 p.m.
Poster Spotlight "How does Weight Sharing Help in Neural Architecture Search?" (Poster Spotlight)   
Yuge Zhang
Fri 12:04 p.m. - 12:06 p.m.
Poster Spotlight "Rethinking NAS Operations for Diverse Tasks" (Poster Spotlight)   
Nicholas Roberts, Mikhail Khodak
Fri 12:06 p.m. - 12:08 p.m.
Poster Spotlight "Cost-aware Adversarial Best Arm Identification" (Poster Spotlight)   
Nikita Ivkin, Valerio Perrone, Giovanni Zappella, Zohar Karnin
Fri 12:08 p.m. - 1:10 p.m.
 link »

Please join us in GatherTown for our poster session.

Fri 1:10 p.m. - 1:20 p.m.
Fri 1:20 p.m. - 1:21 p.m.
Live introduction for Jeff Clune (Live intro)
Fri 1:21 p.m. - 1:56 p.m.
Invited Talk Jeff Clune (Invited Talk)   
Jeff Clune
Fri 1:56 p.m. - 2:10 p.m.
Q&A Jeff Clune (Q&A Session)
Fri 2:10 p.m. - 2:55 p.m.
Panel Discussion (Discussion Panel)
Fri 2:55 p.m. - 3:00 p.m.
Closing Remarks