Workshop on Theory of Deep Learning: Where next?

The event was live-streamed.

Organizers:

The workshop was organized by Sanjeev Arora (IAS/Princeton University), Joan Bruna (IAS/NYU), Rong Ge (IAS/Duke), Suriya Gunasekar(IAS/Toyota Technical Institute), Jason Lee (IAS/USC), Bin Yu (IAS/UC Berkeley)

This workshop sought to bring together deep learning practitioners and theorists to discuss progress that has been made on deep learning theory, and to identify promising avenues where theory is possible and useful. There were several invited talks each day and also spotlight talks by young researchers.

The workshop was free of charge thanks to the support from the Institute for Advanced Study and the Schwab Charitable Fund made possible by the generosity of Eric and Wendy Schmidt.

Invited Speakers who confirmed participation:

Anima Anandkumar, Raman Arora, Sanjeev Arora, Mikhail Belkin, Léon Bottou, Joan Bruna, Michael Collins, Simon Du, Gintare Karolina Dziugaite, Surya Ganguli, Rong Ge, Suriya Gunasekar, Stefanie Jegelka, Chi Jin, Sham Kakade, Yann LeCun, Jason Lee, Ke Li, Tengyu Ma, Aleksander Madry, Chris Manning, Behnam Neyshabur, Dan Roy, Nathan Sbrero, Rachel Ward, Bin Yu

Agenda:(all talks were in Wolfensohn Hall)

October 15, 2019

REGISTRATION: Opening at 8:30 am - 9:30 am in Wolfensohn Hall and will remain open until 3 pm

Workshop on Theory of Deep Learning: Where next?
Topic: Emergent linguistic structure in deep contextual neural word representations slides video
Speaker: Chris Manning, Stanford University
Time/Room: 9:30am - 10:10am/Wolfensohn Hall
   
Topic: Explaining landscape connectivity of low-cost solutions for multilayer nets video
Speaker: Rong Ge, Duke University; Member, School of Mathematics
Time/Room: 10:10am - 10:40am/Wolfensohn Hall
   
Topic: Fixing GAN optimization through competitive gradient descent video
Speaker: Anima Anandkumar, Caltech
Time/Room: 11:10am - 11:50am/Wolfensohn Hall
   
Topic: Tightening information-theoretic generalization bounds with data-dependent estimates with an application to SGLD video
Speaker: Daniel Roy, University of Toronto
Time/Room: 11:50am - 12:20pm/Wolfensohn Hall
   
Topic: Spotlight Talks: Yuanzhi Li, Soham De, Mahyar Fazlyab, Maithra Raghu, Valentin Thomas video
Speaker: Various
Time/Room: 12:20pm - 1:00 pm
   
Topic: Is optimization the right language to understand deep learning? video
Speaker: Sanjeev Arora, Princeton University; Distinguished Visiting Professor, School of Mathematics
Time/Room: 2:30pm - 3:10pm/Wolfensohn Hall
   
Topic: Spotlight Talks: Amir Asadi, Dimitris Kalimeris video
Speaker: Various
Time/Room: 3:10pm - 3:40pm/Wolfensohn Hall
   
Topic: PAC-Bayesian approaches to understanding generalization in deep learning video slides
Speaker: Gintare Karolina Dziugaite, Simons Institute for the Theory of Computing
Time/Room: 4:00pm - 4:30pm/Wolfensohn Hall
   
Topic: Overcoming the Curse of Dimensionality and Mode Collapse video slides
Speaker: Ke Li, University of California, Berkeley
Time/Room: 4:30pm - 5:00pm/Wolfensohn Hall
   
Topic: Are All Features Created Equal? video slides
Speaker: Aleksander Madry, Massachusetts Institute of Technology
Time/Room: 5:00pm - 5:40pm/Wolfensohn Hall

October 16, 2019

   
Topic: Energy-based Approaches to Representation Learning slides video
Speaker: Yann LeCun, NYU and Facebook AI
Time/Room: 9:30am - 10:10am/Wolfensohn Hall
   
Topic: On Large Deviation Principles for Large Neural Networks video slides
Speaker: Joan Bruna, New York University
Time/Room: 10:10am - 10:40am/Wolfensohn Hall
   
Topic: Neural Models for Speech and Language: Successes, Challenges, and the Relationship to Computational Models of the Brain video
Speaker: Michael Collins, Columbia University
Time/Room: 11:10am - 11:50am/Wolfensohn Hall
   
Topic: On the Connection between Neural Networks and Kernels: a Modern Perspective video
Speaker: Simon Du, Member, School of Mathematics
Time/Room: 11:50am - 12:20pm/Wolfensohn Hall
   
Topic: Hike in Institute Woods
  12:20 pm-1:00 pm
   
Topic: From Classical Statistics to Modern ML: the Lessons of Deep Learning video slides
Speaker: Mikhail Belkin, Ohio State University
Time/Room: 2:30pm - 3:10pm/Wolfensohn Hall
   
Topic: Spotlight Talks: Vaishnavh Nagarajan, Preetum Nakkiran, Xiaowu Dai, Weijie Su video
Speaker: Various
Time/Room: 3:10pm-3:40pm/Wolfensohn Hall
   
Topic: Towards a theoretical foundation of neural networks video
Speaker: Jason Lee, Princeton University; Member, School of Mathematics
Time/Room: 4:00pm - 4:30pm/Wolfensohn Hall
   
Topic: Panel Session
Time/Room: 4:30pm - 5:30pm/Wolfensohn Hall

October 17, 2019

   
Topic: Learning Representations Using Causal Invariance video
Speaker: Leon Bottou, Facebook AI Research
Time/Room: 9:30am - 10:10am/Wolfensohn Hall
   
Topic: Understanding the inductive bias due to dropout video
Speaker: Raman Arora, Johns Hopkins University; Member, School of Mathematics
Time/Room: 10:10am - 10:40am/Wolfensohn Hall
   
Topic: Interpreting Deep Neural Networks video slides
Speaker: Bin Yu, University of California, Berkeley
Time/Room: 11:10am - 11:50am/Wolfensohn Hall
   
Topic: Designing explicit regularizers for deep models video slides
Speaker: Tengyu Ma
Time/Room: 11:50am - 12:20pm/Wolfensohn Hall
   
Topic: Spotlight Talks: Arjun Nitin Bhagoji, Jiaoyang Huang, Rosemary Ke, Or Sharir, Omar Shehab
Speaker: Various video
Time/Room: 12:20pm - 1:00pm/Wolfensohn Hall
   
Topic: Kernel and Rich Regimes in Deep Learning video
Speaker: Nati Srebro, TTIC
Time/Room: 2:30pm - 3:10pm/Wolfensohn Hall
   
Topic: Spotlight Talks: Sebastian Goldt video
Speaker: Various
Time/Room: 3:10pm - 3:40pm/Wolfensohn Hall
   
Topic: Provably Efficient Reinforcement Learning with Linear Function Approximation video
Speaker: Chi Jin, Member, School of Mathematics
Time/Room: 4:00pm - 4:30pm/Wolfensohn Hall
   
Topic: Poster Session
Speaker: Various
Time/Room: 4:30pm - 5:30pm/Wolfensohn Hall

October 18, 2019

   
Topic: Reinforcement Learning, Deep Learning,
and the Role of Policy Gradient Methods video
Speaker: Sham Kakade, University of Washington
Time/Room: 9:30am - 10:10 am/Wolfensohn Hall
   
Topic: Statistical Mechanics of Machine Learning video
Speaker: Surya Ganguli, Stanford University
Time/Room: 10:10am - 10:40am/Wolfensohn Hall
   
Topic: Concentration inequalities for random matrix products
Speaker: Rachel Ward, The University of Texas at Austin; von Neumann Fellow, School of Mathematics
Time/Room: 11:10am - 11:50am/Wolfensohn Hall
   
Topic: Representational Power of Graph Neural Networks video slides
Speaker: Stefanie Jegelka, Massachusetts Institute of Technology
Time/Room: 11:50am - 12:20pm/Wolfensohn Hall
   
Topic: Spotlight Talks: Zhiyuan Li, John Zarka, Stanislav Fort video
Speaker: Various
Time/Room: 12:20pm - 1:00pm/Wolfensohn Hall
   
Topic: Toward a Causal Analysis of Generalization in Deep Learning video
Speaker: Behnam Neyshabur, Google
Time/Room: 2:30pm - 3:00pm/Wolfensohn Hall
   
Topic: Spotlight Talks: Zhifeng Kong, Daniel Paul Kunin, Omar Montasser video
Speaker: Various
Time/Room: 3:10pm - 3:40pm/Wolfensohn Hall
   
Topic: Informal discussion sessions
Time/Room: 3:40pm - 5:30pm/Wolfensohn Hall

~end

 

Contributed talks: Deadline was Sept 2

A few shorter slots were given to showcase late-breaking results and work by young researchers (grads and postdocs).

Selected papers were invited as either talks or posters. Notification deadline was Sept 10.

Date & Time

October 15, 2019 | 9:00am – October 18, 2019 | 6:00am

Categories