Workshops

Cognitive science and artificial intelligence (AI) have a long-standing shared history. Early research in AI was inspired by human intelligence and shaped by cognitive scientists (e.g., Elman, 1990; Rumelhart and McClelland, 1986). At the same time, efforts in understanding human learning and processing used methods and data borrowed from AI to build cognitive models that mimicked human cognition (e.g., Anderson, 1975; Tenenbaum et al., 2006; Lieder & Griffiths, 2017; Dupoux, 2018). In the last five years the field of AI has grown rapidly due to the success of large-scale deep learning models in a variety of applications (such as speech recognition and image classification). Interestingly, algorithms and architectures in these models are often loosely inspired by natural forms of cognition (such as convolutional architectures and experience replay; e.g. Hassabis et al., 2017). In turn, the improvement of these algorithms and architectures enabled more advanced models of human cognition that can replicate, and therefore enlighten our understanding of, human behavior (Yamins & DiCarlo, 2016; Fan et al., 2018; Banino et al., 2018; Bourgin et al., 2019). Empirical data from cognitive psychology has also recently played an important role in measuring how current AI systems differ from humans and in identifying their failure modes (e.g., Linzen et al., 2016; Lake et al., 2017; Gopnik, 2017; Nematzadeh et al., 2018; Peterson et al., 2019; Hamrick, 2019).

The recent advancements in AI confirm the success of a multidisciplinary approach inspired by human cognition. However, the large body of literature supporting each field makes it more difficult for researchers to engage in multidisciplinary research without collaborating. Yet, outside domain-specific subfields, there are few forums that enable researchers in AI to actually connect with people from the cognitive sciences and form such collaborations. Our workshop aims to inspire connections between AI and cognitive science across a broad set of topics, including perception, language, reinforcement learning, planning, human-robot interaction, animal cognition, child development, and reasoning.

Neural Architecture Search (NAS) can be seen as the logical next step in automating the learning of representations. It follows upon the recent transition from manual feature engineering to automatically learning features (using a fixed neural architecture) by replacing manual architecture engineering with automated architecture design. NAS can be seen as a subfield of automated machine learning and has significant overlap with hyperparameter optimization and meta-learning. NAS methods have already outperformed manually designed architectures on several tasks, such as image classification, object detection or semantic segmentation. They have also already found architectures that yield a better trade-off between resource consumption on target hardware and predictive performance. The goal of this workshop is to bring together researchers from industry and academia that focus on NAS. NAS is an extremely hot topic of large commercial interest, and as such has a bit of a history of closed source and competition. It is therefore particularly important to build a community behind this research topic, with collaborating researchers that share insights, code, data, benchmarks, training pipelines, etc, and together aim to advance the science behind NAS.

Differential equations and neural networks are not only closely related to each other but also offer complementary strengths: the modelling power and interpretability of differential equations, and the approximation and generalization power of deep neural networks. The great leap forward in machine learning empowered by deep neural networks has been primarily relying on the increasing amounts of data coupled with modern abstractions of distributed computing. When the models and problems grow larger and more complex, the need for ever larger datasets becomes a bottleneck.

Differential equations have been the principled way to encode prior structural assumptions into nonlinear models such as deep neural networks, reducing their need for data while maintaining the modelling power. These advantages allow the models to scale up to bigger problems with better robustness and safety guarantee in practical settings.

While progress has been made on combining differential equations and deep neural networks, most existing work has been disjointed, and a coherent picture has yet to emerge. Substantive progress will require a principled approach that integrates ideas from the disparate lens, including differential equations, machine learning, numerical analysis, optimization, and physics.

The goal of this workshop is to provide a forum where theoretical and experimental researchers of all stripes can come together not only to share reports on their progress but also to find new ways to join forces towards the goal of coherent integration of deep neural networks and differential equations. Topics to be discussed include, but are not limited to:
- Deep learning for high dimensional PDE problems
- PDE and stochastic analysis for deep learning
- PDE and analysis for new architectures
- Differential equations interpretations of first order optimization methods
- Inverse problems approaches to learning theory
- Numerical tools to interface deep learning models and ODE/PDE solver

Accepted Papers

Contributed Talks
1. Solving ODE with Universal Flows: Approximation Theory for Flow-Based Models. Chin-Wei Huang, Laurent Dinh, Aaron Courville (Paper, Slides).
2. Neural Operator: Graph Kernel Network for Partial Differential Equations. Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar (Paper, Slides).
3. A Mean-field Analysis of Deep ResNet and Beyond:Towards Provable Optimization Via Overparameterization From Depth. Yiping Lu, Chao Ma, Yulong Lu, Jianfeng Lu, Lexing Ying (Paper, Slides).
4. A Free-Energy Principle for Representation Learning. Pratik Chaudhari, Yansong Gao (Paper, Slides).
5. Amortized Finite Element Analysis for Fast PDE-Constrained Optimization. Tianju Xue, Alex Beatson, Sigrid Adriaenssens, Ryan P. Adams (Paper, Slides).
6. Nano-Material Configuration Design with Deep Surrogate Langevin Dynamics. Thanh V. Nguyen, Youssef Mroueh, Samuel Hoffman, Payel Das, Pierre Dognin, Giuseppe Romano, Chinmay Hegde (Paper, Slides).

Poster Lightning Talks 1
7. Nonlinear Differential Equations with External Forcing. Paul Pukite (Paper, Slides).
8. On the space-time expressivity of ResNets. Johannes Christoph Müller (Paper, Slides).
9. Enforcing Physical Constraints in CNNs through Differentiable PDE Layer. Chiyu “Max” Jiang, Karthik Kashinath, Prabhat, Philip Marcus (Paper, Slides).
10. Deep Ritz revisited. Johannes Müller, Marius Zeinhofer (Paper, Slides).
11. Differential Equations as Model Prior for DeepLearning and Applications to Robotics. Michael Lutter, Jan Peters (Paper, Slides).
12. Differentiable Physics Simulation. Junbang Liang, Ming C. Lin (Paper, Slides).
13. Comparing recurrent and convolutional neural networks for predicting wave propagation. Stathi Fotiadis, Eduardo Pignatelli, Mario Lino Valencia, Chris Cantwell, Amos Storkey, Anil A. Bharath (Paper, Slides).
14. Time Dependence in Non-Autonomous Neural ODEs. Jared Quincy Davis, Krzysztof Choromanski, Vikas Sindhwani, Jake Varley, Honglak Lee, Jean-Jacques Slotine, Valerii Likhosterov, Adrian Weller, Ameesh Makadia (Paper, Slides).
15. Learning To Solve Differential Equations Across Initial Conditions. Shehryar Malik, Usman Anwar, Ali Ahmed, Alireza Aghasi (Paper, Slides).
16. How Chaotic Are Recurrent Neural Networks?. Pourya Vakilipourtakalou, Lili Mou (Paper, Slides).
17. Learning-Based Strong Solutions to Forward and Inverse Problems in PDEs. Leah Bar, Nir Sochen (Paper, Slides).
18. Embedding Hard Physical Constraints in Convolutional Neural Networks for 3D Turbulence. Arvind T. Mohan, Nicholas Lubbers, Daniel Livescu, Michael Chertkov (Paper, Slides).
19. Wavelet-Powered Neural Networks for Turbulence. Arvind T. Mohan, Daniel Livescu, Michael Chertkov (Paper, Slides).
20. Can auto-encoders help with filling missing data?. Marek Śmieja, Maciej Kołomycki, Łukasz Struski, Mateusz Juda, Mário A. T. Figueiredo (Paper, Slides).
21. Neural Differential Equations for Single Image Super-Resolution. Teven Le Scao (Paper, Slides).
22. Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View. Yiping Lu, Zhuohan Li, Di He, Zhiqing Sun, Bin Dong, Tao Qin, Liwei Wang, Tie-yan Liu (Paper, Slides).
23. Neural Dynamical Systems. Viraj Mehta, Ian Char, Willie Neiswanger, Youngseog Chung, Andrew Oakleigh Nelson, Mark D Boyer, Jeff Schneider (Paper, Slides).

Poster Lightning Talks 2
24. Progressive Growing of Neural ODEs. Hammad A. Ayyubi, Yi Yao, Ajay Divakaran (Paper, Slides).
25. Fast Convergence for Langevin with Matrix Manifold Structure. Ankur Moitra, Andrej Risteski (Paper, Slides).
26. Bringing PDEs to JAX with forward and reverse modes automatic differentiation. Ivan Yashchuk (Paper, Slides).
27. Urban air pollution forecasts generated from latent space representation. Cesar Quilodran Casas, Rossella Arcucci, Yike Guo (Paper, Slides).
28. Dissipative SymODEN: Encoding Hamiltonian Dynamics with Dissipation and Control into Deep Learning. Yaofeng Desmond Zhong, Biswadip Dey, Amit Chakraborty (Paper, Slides).
29. Neural Ordinary Differential Equation Value Networks for Parametrized Action Spaces. Michael Poli, Stefano Massaroli, Sanzhar Bakhtiyarov, Atsushi Yamashita, Hajime Asama, Jinkyoo Park (Paper, Slides).
30. Stochasticity in Neural ODEs: An Empirical Study. Alexandra Volokhova, Viktor Oganesyan, Dmitry Vetrov (Paper, Slides).
31. Generating Control Policies for Autonomous Vehicles Using Neural ODEs. Houston Lucas, Richard Kelley (Paper, Slides).
32. Generative ODE Modeling with Known Unknowns. Ori Linial, Uri Shalit (Paper, Slides).
33. Encoder-decoder neural network for solving the nonlinear Fokker-Planck-Landau collision operator in XGC. Marco Andres Miller, Randy Michael Churchill, Choong-Seock Chang, Robert Hager (Paper, Slides).
34. Differentiable Molecular Simulations for Control and Learning. Wujie Wang, Simon Axelrod, Rafael Gómez-Bombarelli (Paper, Slides).
35. Port-Hamiltonian Gradient Flows. Michael Poli, Stefano Massaroli, Atsushi Yamashita, Hajime Asama, Jinkyoo Park (Paper, Slides).
36. Lagrangian Neural Networks. Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, Shirley Ho (Paper, Slides).
37. Constrained Neural Ordinary Differential Equations with Stability Guarantees. Aaron Tuor, Jan Drgona, Draguna Vrabie (Paper, Slides).
38. Stochastic gradient algorithms from ODE splitting perspective. Daniil Merkulov, Ivan Oseledets (Paper, Slides).
39. The equivalence between Stein variational gradient descent and black-box variational inference. Casey Chu, Kentaro Minami, Kenji Fukumizu (Paper, Slides).
40. Towards Understanding Normalization in Neural Ordinary Differential Equations. Julia Gusak, Larisa Markeeva, Talgat Daulbaev, Alexander Katrutsa, Andrzej Cichocki, Ivan Oseledets (Paper, Slides).

Call for Papers and Submission Instructions

We invite researchers to submit anonymous extended abstracts of up to 4 pages (including abstract, but excluding references). No specific formatting is required. Authors may use the ICLR style file, or any other style as long as they have standard font size (11pt) and margins (1in).

Submissions should be anonymous and are handled through the OpenReview system. Please note that at least one coauthor of each accepted paper will be expected to attend the workshop in person to present a poster or give a contributed talk.

Papers can be submitted at the address:

https://openreview.net/group?id=ICLR.cc/2020/Workshop/DeepDiffEq

Important Dates

Submission Deadline (EXTENDED): 23:59 pm PST, Tuesday, February 18th
Acceptance notification: Tuesday, February 25th
Camera ready submission: Sunday, April 19th
Workshop: Sunday, April 26th

ML-IRL will focus on the challenges of real-world use of machine learning and the gap between what ML can do in theory and what is needed in practice. Given the tremendous recent advances in methodology from causal inference to deep learning, the strong interest in applications (in health, climate and beyond), and discovery of problematic implications (e.g. issues of fairness and explainability) now is an ideal time to examine how we develop, evaluate and deploy ML and how we can do it better. We envision a workshop that is focused on productive solutions, not mere identification of problems or demonstration of failures.

Overall, we aim to examine how real-world applications can and should influence every stage of ML, from how we develop algorithms to how we evaluate them. These topics are fundamental for the successful real-world use of ML, but are rarely prioritized. We believe that a workshop focusing on these issues in a domain independent way is a necessary starting point for building more useful and usable ML. We will have speakers and participants representing all core topics (developing novel algorithms that work in the real world, specific applications and how we can learn from them, human factors and fairness) and bringing experience in true real-world deployments (e.g. fighting poverty with data, clinical trials). In addition to building a community and awareness of the research gap, ML-IRL will produce a whitepaper with key open problems and a path toward solutions.

Climate change is one of the greatest problems society has ever faced, with increasingly severe consequences for humanity as natural disasters multiply, sea levels rise, and ecosystems falter. Since climate change is a complex issue, action takes many forms, from designing smart electric grids to tracking greenhouse gas emissions through satellite imagery. While no silver bullet, machine learning can be an invaluable tool in fighting climate change via a wide array of applications and techniques. These applications require algorithmic innovations in machine learning and close collaboration with diverse fields and practitioners. This workshop is intended as a forum for those in the machine learning community who wish to help tackle climate change.

As ML systems are pervasively deployed, security and privacy challenges became central to their design. The community has produced a vast amount of work to address these challenges and increase trust in ML. Yet, much of the work concentrates on well-defined problems that enable nice tractability from a mathematical perspective but are hard to translate to the threats that target real-world systems.

This workshop calls for novel research that addresses the security and privacy risks arising from the deployment of ML, from malicious exploitation of vulnerabilities (e.g., adversarial examples or data poisoning) to concerns on fair, ethical and privacy-preserving uses of data. We aim to provide a home to new ideas “outside the box”, even if proposed preliminary solutions do not match the performance guarantees of known techniques. We believe that such ideas could prove invaluable to more effectively spur new lines of research that make ML more trustworthy.

We aim to bring together experts from a variety of communities (machine learning, computer security, data privacy, fairness & ethics) in an effort to synthesize promising ideas and research directions, as well as foster and strengthen cross-community collaborations. Indeed, many fundamental problems studied in these diverse areas can be broadly recast as questions around the (in-)stability of machine learning models: generalization in ML, model memorization in privacy, adversarial examples in security, model bias in fairness and ethics, etc.

Machine learning has enabled significant improvements in many areas. Most of these ML methods are based on inferring statistical correlations, they can become unreliable where spurious correlations present in the training data do not hold in the testing setting. One way of tackling this problem is to learn the causal structure of the data generating processing (causal models). The general problem of causal discovery requires performing all interventions on the model. However, this may be too expensive and/or infeasible in real environments: understanding how to most efficiently intervene in the environment in order to uncover the most amount of information is therefore a necessary requirement to be able to uncover causal information in real-world applications. In this workshop, we investigate a few key questions or topics.
- What is the role of an underlying causal model in decision making?
- What is the difference between a prediction that is made with a causal model and one made with a non‐causal model?
- What is the role of causal models in decision-making in real-world settings, for example in relation to fairness, transparency, and safety?
- The way current RL agents explore environments appears less intelligent than the way human learners explore. One reason for this disparity might be due to the fact that when faced with a novel environment, humans do not merely observe, but actively interact with the world affecting it through actions. Furthermore, curating a causal model of the world allows the learner to maintain a set of plausible hypotheses and design experiments to test these hypotheses.
- Can we use a distributional belief about the agent's model of the world as a tool for exploration (minimize entropy, maximize knowledge acquisition)?
- Can we learn an incomplete causal model that is sufficient for good decision making as only parts of the model might be relevant for the tasks at hand. How can we efficiently learn these causal sub-models?

Artificial intelligence has invaded the agriculture field during the last few years. From automatic crop monitoring via drones, smart agricultural equipment, food security and camera-powered apps assisting farmers to satellite image based global crop disease prediction and tracking, computer vision has been a ubiquitous tool. This workshop aims to expose the fascinating progress and unsolved problems of computational agriculture to the AI research community. It is jointly organized by AI and computational agriculture researchers and has the support of CGIAR, a global partnership that unites international organizations engaged in agricultural research. The workshop, will feature invited talks, panels and discussions on gender and agriculture in the digital era and AI for food security. It will also host and fund two open, large-scale competitions with prizes as well as a prototyping session.

Recent work has demonstrated that current reinforcement learning methods are able to master complex tasks given enough resources. However, these successes have mainly been confined to single and unchanging environments. By contrast, the real world is both complex and dynamic, rendering it impossible to anticipate each new scenario. Many standard learning approaches require tremendous resources in data and compute to re-train. However, learning also offers the potential to develop versatile agents that adapt and continue to learn across environment changes and shifting goals and feedback. To achieve this, agents must be able to apply knowledge gained in past experience to the situation at hand. We aim to bring together areas of research that provide different perspectives on how to extract and apply this knowledge.
The BeTR-RL workshop aims to bring together researchers from different backgrounds with a common interest in how to extend current reinforcement learning algorithms to operate in changing environments and tasks. Specifically, we are interested in the following lines of work: leveraging previous experience to learn representations or learning algorithms that transfer to new tasks (transfer and meta-learning), generalizing to new scenarios without any explicit adaptation (multi-task and goal-conditioned RL), and learning new capabilities while retaining the previously learned skills (continual learning). The workshop aims at further developing these research directions while determining similarities and trade-offs.

Earth sciences or geosciences encompasses understanding the physical characteristics of our planet, including its lithosphere, hydrosphere, atmosphere and biosphere, applying all fields of natural and computational sciences. As Earth sciences enters the era of increasing volumes and variety of geoscientific data from sensors, as well as high performance computing simulations, machine learning methods are poised to augment and in some cases replace traditional methods. Interest in the application of machine learning, deep learning, reinforcement learning, computer vision and robotics to the geosciences is growing rapidly at major Earth science and machine learning conferences.

Our workshop proposal AI for Earth sciences seeks to bring cutting edge geoscientific and planetary challenges to the fore for the machine learning and deep learning communities. We seek machine learning interest from major areas encompassed by Earth sciences which include, atmospheric physics, hydrologic sciences, cryosphere science, oceanography, geology, planetary sciences, space weather, geo-health (i.e. water, land and air pollution), volcanism, seismology and biogeosciences. We call for papers demonstrating novel machine learning techniques in remote sensing for meteorology and geosciences, generative Earth system modeling, and transfer learning from geophysics and numerical simulations and uncertainty in Earth science learning representations. We also seek theoretical developments in interpretable machine learning in meteorology and geoscientific models, hybrid models with Earth science knowledge guided machine learning, representation learning from graphs and manifolds in spatiotemporal models and dimensionality reduction in Earth sciences. In addition, we seek Earth science applications from vision, robotics and reinforcement learning. New labelled Earth science datasets and visualizations with machine learning is also of particular interest.

Healthcare is under significant pressure: costs are rising, populations are aging, lifestyles are becoming more sedentary, and critically we lack experts to meet the rising demand. In addition, in under-developed countries healthcare quality remains limited. Meanwhile, AI has shown great promise for healthcare applications, and the digitisation of data and use of electronic health records is becoming more widespread. AI could play a key role in enabling, democratising and upholding high standards of healthcare worldwide, assisting health professionals to make decisions faster, more accurately and more consistently.

However, so far, the adoption of AI in real-world healthcare applications has been slow relative to that in domains such as autonomous driving. In this workshop, we aim to highlight recent advances and the potential opportunities, via a series of talks and panel discussion. We are supported by local institutes in Ethiopia, which besides being the ICLR host country, is a prime example of a country that potentially stands to gain much from the application of AI to healthcare. We invite technical papers and white papers addressing challenges which are important to the real-world deployment of AI in healthcare.

The rise in ML community efforts on the African continent has led to a growing interest in Natural Language Processing, particularly for African languages which are typically low resource languages. This interest is manifesting in the form of national, regional, continental and even global collaborative efforts to build corpora, as well as the application of the aggregated corpora to various NLP tasks.

This workshop therefore has several aims;
1) to showcase this work being done by the African NLP community and provide a platform to share this expertise with a global audience interested in NLP techniques for low resource languages
2) to provide a platform for the groups involved with the various projects to meet, interact, share and forge closer collaboration
3) to provide a platform for junior researchers to present papers, solutions, and begin interacting with the wider NLP community
4) to present an opportunity for more experienced researchers to further publicize their work and inspire younger researchers through keynotes and invited talks

According to the World Health Organization (WHO), cancer is the second leading cause of death globally, and was responsible for an estimated 9.6 million deaths in 2018. Approximately 70% of deaths from cancer occur in low- and middle-income countries (LMIC), in large part due to lack of proper access to screening, diagnosis and treatment services. As the economic impact of cancer increases, disparities in diagnosis and treatment options prevail. Recent advances in the field of machine learning have bolstered excitement for the application of assistive technologies in the medical domain, with the promise of improved care for patients. Unfortunately, cancer care in LMIC faces a very different set of challenges, unless focused efforts are made to overcome these challenges, cancer care in these countries will be largely unaffected. The purpose of this workshop is to bring together experts in machine learning and clinical cancer care to facilitate discussions regarding challenges in cancer care and opportunities for AI to make an impact. In particular, there is an immense potential for novel representation learning approaches to learn from different data modalities such as pathology, genomics, and radiology. Studying these approaches have the potential to significantly improve survival outcomes and improve the lives of millions of people.

The constant progress being made in artificial intelligence needs to extend across borders if we are to democratize AI in developing countries. Adapting the state-of-the-art (SOTA) methods to resource constrained environments such as developing countries is challenging in practice. Recent breakthroughs in natural language processing (NLP), for instance, rely on increasingly complex and large models (e.g. most models based on transformers such as BERT, VilBERT, ALBERT, and GPT-2) that are pre-trained in on large corpus of unlabeled data. In most developing countries, low/limited resources means hard path towards adoption of these breakthroughs. Methods such as transfer learning will not fully solve the problem either due to bias in pre-training datasets that do not reflect real test cases in developing countries as well as the prohibitive cost of fine-tuning these large models. Recent progress with focus given to ML for social good has the potential to alleviate the problem in part. However, the themes in such workshops are usually application driven such as ML for healthcare and for education, and less attention is given to practical aspects as it relates to developing countries in implementing these solutions in low or limited resource scenarios. This, in turn, hinders the democratization of AI in developing countries. As a result, we aim to fill the gap by bringing together researchers, experts, policy makers and related stakeholders under the umbrella of practical ML for developing countries. The workshop is geared towards fostering collaborations and soliciting submissions under the broader theme of practical aspects of implementing machine learning (ML) solutions for problems in developing countries. We specifically encourage contributions that highlight challenges of learning under limited or low resource environments that are typical in developing countries.

In this ICLR workshop, we will explore the intersection of big science and AI and the changing nature of good fundamental research in the era of pervasive AI. The workshop will also examine how AI can enhance the social good of Big Science, relevant for the African continent given that the Square Kilometre Array (SKA), which will be one of the largest astronomical endeavors ever undertaken, will be majority hosted in Africa. In addition this workshop aims to stimulate discussion across astronomy, cosmology and particle physics.