Robust and reliable machine learning in the real world

Di Jin, Eric Wong, Yonatan Belinkov, Kai-Wei Chang, Zhijing Jin, Yanjun Qi, Aditi Raghunathan, Tristan Naumann, Mohit Bansal


As machine learning (ML) is deployed pervasively, there is an increasing demand for ML systems to behave reliably when the input to the system has changed. Much work has emerged regarding artificial and natural changes to data, with a growing interest towards studying robustness and reliability of ML systems in the presence of real-world changes. This shift towards more realistic considerations raises both old and new fundamental questions for machine learning:
1. Can we bring principled research in robustness closer to real-world effects?
2. How can we demonstrate the reliability of ML systems in real-world deployments?
3. What are the unique societal and legal challenges facing robustness for deployed ML systems?
Consequently, the goal of this workshop is to bring together research in robust machine learning with the demands and reliability constraints of real-world processes and systems, with a focus on the practical, theoretical, and societal challenges in bringing these approaches to real world-scenarios. We highlight emerging directions, paradigms, and applications which include 1. Characterizing real-world changes for robustness; 2. Reliability of real-world systems; 3. Societal and legal considerations.

Chat is not available.

Timezone: »


Fri 8:05 a.m. - 8:15 a.m.
Opening remarks (Moderation)
Fri 8:16 a.m. - 8:30 a.m.
Contributed Talk 1 - On the Benefits of Defining Vicinal Distributions in Latent Space (Contributed talk)   
Puneet Mangla
Fri 8:31 a.m. - 9:00 a.m.
Ece Kamar - AI in the Open World: Discovering Blind Spots of AI (Invited talk)   
Ece Kamar
Fri 9:00 a.m. - 9:15 a.m.
Ece Kamar Q&A (Q&A)
Fri 9:16 a.m. - 9:45 a.m.
Kendra Albert - Panda v. Gibbon: Legal Liability for Adversarial ML Attacks (Invited talk)   
Kendra Albert
Fri 9:45 a.m. - 10:00 a.m.
Kendra Albert Q&A (Q&A)
Fri 10:00 a.m. - 11:00 a.m.
 link »

Join us for our poster session in gather town!

Fri 11:01 a.m. - 11:15 a.m.
Contributed Talk 2 - Neural Lower Bounds for Verification (Contributed talk)   
Florian Jaeckle
Fri 11:16 a.m. - 11:45 a.m.
Percy Liang - Self-training Algorithms and Analyses for Unsupervised Domain Adaptation (Invited talk)   
Percy Liang
Fri 11:45 a.m. - 12:00 p.m.
Percy Liang Q&A (Q&A)
Fri 12:00 p.m. - 1:00 p.m.
Ece Kamar, Finale Doshi-Velez, Kendra Albert, Nicolas Papernot (live) (Panel discussion)
Fri 1:00 p.m. - 2:00 p.m.
Fri 2:01 p.m. - 2:15 p.m.
Contributed Talk 3 - Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks (Contributed talk)   
Curtis G Northcutt
Fri 2:16 p.m. - 2:45 p.m.
Ruoxi Jia (live) (Invited talk)
Ruoxi Jia
Fri 2:46 p.m. - 3:00 p.m.
Contributed Talk 4 - A Causal Lens for Controllable Text Generation (Contributed talk)
Zhiting Hu
Fri 3:00 p.m. - 4:00 p.m.
 link »

Join us for our poster session in Gather Town!

Fri 4:01 p.m. - 4:15 p.m.
Contributed Talk 5 - On Calibration and Out-of-Domain Generalization (Contributed talk)   
Yoav Wald
Fri 4:16 p.m. - 4:45 p.m.
Bo Li - Secure Learning in Adversarial Environments with Knowledge Inference (Invited talk)   
Bo Li
Fri 4:45 p.m. - 5:00 p.m.
Bo Li Q&A (Q&A)
Fri 5:00 p.m. - 5:05 p.m.
Closing remarks (Moderation)
Eric Wong