ICLR 2015

May 7 - 9, 2015

The Hilton San Diego Resort & Spa

There is a negotiated room rate for ICLR 2015. Please use this link for reservations. If you have difficulty with the booking site, please call the Hilton San Diego's in-house reservation team directly at +1-619-276-4010 ext. 1.

Anyone registering after April 29, 2015 will need to see Karen Smith at the registration desk for a badge.

Note that the registration fee includes breakfast, coffee breaks, dinner, and the joint ICLR/AISTATS reception. See the conference schedule for the timing of these events.

Online Registration Form

Yoshua Bengio, Université de Montreal
Yann LeCun, New York University and Facebook

Brian Kingsbury, IBM Research
Samy Bengio, Google
Nando de Freitas, University of Oxford
Hugo Larochelle, Université de Sherbrooke

iclr2015.programchairs@gmail.com

https://www.facebook.com/iclr.cc

ICLR 2015 gratefully acknowledges the support of its sponsors.

network: Hilton Resort
username: iclr2015
password: deeplearning

Artificial Tasks for Artificial Intelligence

Despite great recent advances, the road towards intelligent machines able to reason and adapt in real-time in multimodal environments remains long and uncertain. This final goal is so complex and further away that it is impossible to perform experiments and research directly in the desired final conditions, so one has to use intermediate and/or proxy tasks as midway goals. Some of those tasks like object detection in computer vision, or machine translation in natural language processing are very useful on their own and fuel many applications. However, such intermediate tasks are already very difficult and it is not obvious that they are suited testbeds for designing intelligent systems: their inherent complexity makes it hard to precisely interpret the behavior and true capabilities of algorithms, in particular regarding key sophisticated capabilities like reasoning and planning. Hence, in this talk, we advocate the use of controlled artificial environments for developing research in AI, environments in which one can precisely study the behavior of algorithms and unambiguously assess their abilities.

This talk follows from joint work and discussions with Jason Weston, Sumit Chopra, Tomas Mikolov and Leon Bottou, among others.

Deep Reinforcement Learning

In this talk I will discuss how reinforcement learning (RL) can be combined with deep learning (DL). There are several ways to combine DL and RL together, including value-based, policy-based, and model-based approaches with planning. Several of these approaches have well-known divergence issues, and I will present simple methods for addressing these instabilities. These methods have achieved notable success in the Atari 2600 domain. I will present recent a selection of recent results that improve on the published state-of-the-art in Atari and other challenging domains. Finally, I will discuss how RL can be used to improve DL, even when the native problem is supervised or unsupervised learning.

Beyond Representation Learning

As we build ever deeper networks with ever more sophisticated representations it is a good time to pause and ask ourselves where this will end. Building ever taller skyscrapers gets our heads in the clouds but will it get us to the moon? A good place to look for answers is nature. This lecture will start with a look at the hierarchy of cortical areas where much of our intuition about deep learning came from, and will explore essential brain regions that these cortical areas communicate with that give rise to intelligent behavior.

Learning Latent Programs for Question Answering

“The first Summer Olympics that had at least 20 nations took place in which city?” We tackle the problem of building a system to answering these questions that involve computing the answer. We propose a methodology based on semantic parsing, where we map a question onto a latent program (logical form), whose execution yields the answer (denotation). To obtain both depth (complexity of the program) and breadth (diversity of the questions/domains), we define a new task of answering a complex question from semi-structured tables on the web. We show promising results on the new dataset and invite the community to take on this challenge.

Algorithms that Learn to Think on their Feet

The classic framework of machine learning is: example in, prediction out. This is great when examples are fully available. But it is very different from how humans reason. We get some information and may make a prediction. Or we may decide to get more information. For us, it's worth spending effort when making hard and important decisions (e.g., foreign policy); it is not on easy or low-cost decisions (e.g., afternoon snacks).

I'll describe our recent work that focuses on information cost, value, and time. I'll show examples from three settings in natural language processing: syntactic parsing, question answering in competitions and simultaneous machine translation. The last is the problem of incrementally producing a translation of a foreign sentence before the entire sentence is “heard” and is challenging even for well-trained humans.

This is joint work with a number of fantastic collaborators: Jordan Boyd-Graber, Leonardo Claudino, Jason Eisner, Lise Getoor, Alvin Grissom II, He He, Mohit Iyyer, John Morgan, Jay Pujara and Richard Socher.

The Ebb and Flow of Deep Learning: a Theory of Local Learning

In a physical neural system, where storage and processing are intertwined, the learning rules for adjusting synaptic weights can only depend on local variables, such as the activity of the pre- and post-synaptic neurons. Thus learning models must specify two things: (1) which variables are to be considered local; and (2) which kind of function combines these local variables into a learning rule. We consider polynomial learning rules and analyze their behavior and capabilities in both linear and non-linear networks. As a byproduct, this framework enables the discovery of new learning rules and important relationships between learning rules and group symmetries.

Stacking local learning rules in deep feedforward networks leads to deep local learning. While deep local learning can learn interesting representations, it cannot learn complex input-output functions, even when targets are available for the top layer. Learning complex input-output functions requires instead local deep learning, where target information is transmitted to the deep layers, thereby raising two fundamental issues: (1) the nature of the transmission channel; and (2) the nature and amount of information transmitted over this channel. This leads to the class of deep targets learning algorithms, which provide targets for the deep layers, and its stratification along the information spectrum, illuminating the remarkable power and uniqueness of the backpropation algorithm. The theory clarifies the concept of Hebbian learning, what is learnable by Hebbian learning, and explains the sparsity of the space of learning rules discovered so far and the unique role backpropagation plays in this space.

• Word Representations via Gaussian Embedding, Luke Vilnis and Andrew McCallum
• Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN), Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, and Alan Yuille
• Deep Structured Output Learning for Unconstrained Text Recognition, Max Jaderberg, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman
• Very Deep Convolutional Networks for Large-Scale Image Recognition, Karen Simonyan and Andrew Zisserman
• Fast Convolutional Nets With fbfft: A GPU Performance Evaluation, Nicolas Vasilache, Jeff Johnson, Michael Mathieu, Soumith Chintala, Serkan Piantino, and Yann LeCun
• Reweighted Wake-Sleep, Jorg Bornschein and Yoshua Bengio
• The local low-dimensionality of natural images, Olivier Henaff, Johannes Balle, Neil Rabinowitz, and Eero Simoncelli
• Memory Networks, Jason Weston, Sumit Chopra, and Antoine Bordes
• Object detectors emerge in Deep Scene CNNs, Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba
• Qualitatively characterizing neural network optimization problems, Ian Goodfellow and Oriol Vinyals
• Neural Machine Translation by Jointly Learning to Align and Translate, Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio

* Each oral has a 20-minute time slot. Please prepare 15 minutes of material, and plan to use the last 5 minutes for questions and switching between speakers.

* The poster boards are 4' high x 8' wide (120 cm high X 240 cm wide).