Workshop on Theory of Deep Learning: Where next?

2019-2020
Tuesday, October 15, 2019 - 09:00 to Friday, October 18, 2019 - 06:00

Organizers:

The workshop is organized by Sanjeev Arora (IAS/Princeton University), Joan Bruna (IAS/NYU), Rong Ge (IAS/Duke), Suriya Gunasekar(IAS/Toyota Technical Institute), Jason Lee (IAS/USC), Bin Yu (IAS/UC Berkeley)

This workshop seeks to bring together deep learning practitioners and theorists to discuss progress that has been made on deep learning theory, and to identify promising avenues where theory is possible and useful. We will have several invited talks each day and also spotlight talks by young researchers.

Invited Speakers who have confirmed participation:

Anima Anandkumar, Raman Arora, Sanjeev Arora, Mikhail Belkin, Léon Bottou, Joan Bruna, Simon Du, Surya Ganguli, Rong Ge, Suriya Gunasekar, Stefanie Jegelka, Sham Kakade, Yann LeCun, Jason Lee, Ke Li, Tengyu Ma, Aleksander Madry, Chris Manning, Behnam Neyshabur, Karolina Dziugaite Roy, Dan Roy, Nathan Sbrero, Rachel Ward, Bin Yu

Registration: click here

If you have any problems with registration or other practical questions, please write to Michelle Huguenin (huguenin@ias.edu).

Registration for the meals and workshop are required. (Speakers do not need to register)  

Although there is no registration fee, because seating is limited both in the seminar room and in our dining hall, you are required to register if you wish to attend all or part of the workshop.  In addition, in order for our chef to prepare enough food, we need to have a headcount for the meals.  Please note the workshop attendees are expected to pay for their lunch. Approximate range of prices are 4.00 for a salad to 9.00 for an entree. Vegetarian lunches will be available.  Only cash is accepted at the cash registers.  Credit cards and personal checks are not accepted.

Agenda:(all talks are in Wolfensohn Hall)

October 15, 2019

Workshop on Theory of Deep Learning: Where next?
Topic: Emergent linguistic structure in deep contextual neural word representations
Speaker: Chris Manning, Stanford University
Time/Room: 9:30am - 10:10am/Wolfensohn Hall
Topic: What 2-layer neural nets can we optimize?
Speaker: Rong Ge, Duke University; Member, School of Mathematics
Time/Room: 10:10am - 10:40am/Wolfensohn Hall
Topic: Fixing GAN optimization through competitive gradient descent
Speaker: Anima Anandkumar, Caltech
Time/Room: 11:10am - 11:50am/Wolfensohn Hall
Topic:  PAC-Bayesian approaches to understanding generalization in deep learning
Speaker: Karolina Dziugaite Roy, Simons Institute for the Theory of Computing
Time/Room: 11:50am - 12:20pm/Wolfensohn Hall
Topic: Spotlight Talks
Speaker: Various
Time/Room: 12:20pm - 1:00 pm
Topic: TBA
Speaker: Sanjeev Arora, Princeton University; Distinguished Visiting Professor, School of Mathematics
Time/Room: 2:30pm - 3:10pm/Wolfensohn Hall
Topic: Spotlight Talks
Speaker: Various
Time/Room: 3:10pm - 3:40pm/Wolfensohn Hall
Topic: Tightening information-theoretic generalization bounds with data-dependent estimates with an application to SGLD
Speaker: Daniel Roy, University of Toronto
Time/Room: 4:00pm - 4:30pm/Wolfensohn Hall
Topic: Overcoming the Curse of Dimensionality and Mode Collapse
Speaker: Ke Li, University of California, Berkeley
Time/Room: 4:30pm - 5:00pm/Wolfensohn Hall
Topic: TBA
Speaker: Behnam Neyshabur, Google
Time/Room: 5:00pm - 5:30pm/Wolfensohn Hall

October 16, 2019

Topic: TBA
Speaker: Yann LeCun, NYU and Facebook AI
Time/Room: 9:30am - 10:10am/Wolfensohn Hall
Topic: On Large Deviation Principles for Large Neural Networks
Speaker: Joan Bruna, New York University
Time/Room: 10:10am - 10:40am/Wolfensohn Hall
Topic:
Neural Models for Speech and Language: Successes, Challenges, and the Relationship to Computational Models of the Brain
Speaker: Michael Collins, Columbia University
Time/Room: 11:10am - 11:50am/Wolfensohn Hall
Topic:
On the Connection between Neural Networks and Kernels: a Modern Perspective
Speaker: Simon Du, Member, School of Mathematics
Time/Room: 11:50am - 12:20am/Wolfensohn Hall
Topic:
From Classical Statistics to Modern ML: the Lessons of Deep Learning
Speaker: Mikhail Belkin, Ohio State University
Time/Room: 2:30pm - 3:10pm/Wolfensohn Hall
Topic: Spotlight Talks
Speaker: Various
Time/Room: 3:10pm-3:40pm/Wolfensohn Hall
Topic: TBA
Speaker: Jason Lee, Princeton University; Member, School of Mathematics
Time/Room: 4:00pm - 4:30pm/Wolfensohn Hall
Topic: Panel Session
Time/Room: 4:30pm - 5:30pm/Wolfensohn Hall

October 17, 2019

Topic: Learning Representations Using Causal Invariance
Speaker: Leon Bottou, Facebook AI Research
Time/Room: 9:30am - 10:10am/Wolfensohn Hall
Topic: Understanding the inductive bias due to dropout
Speaker: Raman Arora, Johns Hopkins University; Member, School of Mathematics
Time/Room: 10:10am - 10:40am/Wolfensohn Hall
Topic:
Interpreting Deep Neural Networks
Speaker: Bin Yu, University of California, Berkeley
Time/Room: 11:10am - 11:50am/Wolfensohn Hall
Topic:
Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks
Speaker: Tengyu Ma
Time/Room: 11:50am - 12:20pm/Wolfensohn Hall
Topic: Spotlight Talks
Speaker: Various
Time/Room: 12:20pm - 1:00pm/Wolfensohn Hall
Topic: TBA
Speaker: Nati Srebro, TTIC
Time/Room: 2:30pm - 3:10pm/Wolfensohn Hall 
Topic: Spotlight Talks
Speaker: Various
Time/Room: 3:10pm - 3:40pm/Wolfensohn Hall
Topic: Provably Efficient Reinforcement Learning with Linear Function Approximation
Speaker: Chi Jin, University of California, Berkeley
Time/Room: 4:00pm - 4:30pm/Wolfensohn Hall
Topic: Poster Session
Speaker: Various
Time/Room: 4:30pm - 5:30pm/Wolfensohn Hall

October 18, 2019

Topic: TBA
Speaker: Sham Kakade, University of Washington
Time/Room: 9:30am - 10:10 am/Wolfensohn Hall
Topic: TBA
Speaker: Surya Ganguli, Stanford University
Time/Room: 10:10am - 10:40am/Wolfensohn Hall
Topic: On adaptivity and generalization in deep learning
Speaker: Rachel Ward, The University of Texas at Austin; von Neumann Fellow, School of Mathematics
Time/Room: 11:10am - 11:50am/Wolfensohn Hall
Topic: TBA
Speaker: Stefanie Jegelka, Massachusetts Institute of Technology
Time/Room: 11:50am - 12:20pm/Wolfensohn Hall
Topic: Spotlight Talks
Speaker: Various
Time/Room: 12:20pm - 1:00pm/Wolfensohn Hall
Topic:
Are All Features Created Equal
Speaker: Aleksander Madry, Massachusetts Institute of Technology
Time/Room: 2:30pm - 3:10pm/Wolfensohn Hall
Topic: Spotlight Talks
Speaker: Various
Time/Room: 3:10pm - 3:40pm/Wolfensohn Hall
Topic: Informal discussion sessions
Time/Room: 3:40pm - 5:30pm/Wolfensohn Hall

~end

 

Contributed talks: Deadline Sept 2

A few shorter slots are being reserved to showcase late-breaking results and work by young researchers (grads and postdocs). To be considered please send an email to iasdlworkshop@gmail.com by Sept 2 with an abstract in the body of the email and pdf of the paper either as attachment or as arxiv link.

Selected papers will be invited as either talks or posters. Notification deadline: Sept 10.

 

Travel Information:

Directions to IAS (driving, train and plane):  https://www.ias.edu/about/maps-directions

Places to Stay:

  • The Peacock Inn (1.5 miles from IAS) 20 Bayard Lane, Pirnceton, NJ 08540 - 609-924-1707
  • Nassau Inn (1.6 miles from IAS) 10 Palmer Square, Princeton, NJ 08542 - 609-921-7500
  • Hyatt Regency (3.1 miles from IAS) 102 Carnegie Center Drive, Princeton, NJ 08540 - 609-987-1234
  • Marriott Residence Inn (3.7 miles from IAS) 3563 US Route 1, Princeton, NJ 08540 - 609-799-0550
  • Hyatt Place (3.8 miles from IAS) 3565 US Highway 1, Princeton, NJ 08540 - 609-720-0200
  • Extended Stay America (4.4 miles from IAS) 3450 US Highway 1, Princeton, NJ 08540 - 609-919-9000
  • Courtyard Marriott (5.3 Miles from IAS) 3815 US Highway 1 at Mapleton Road, Princeton, NJ 08540 - 609-716-9100
  • Homewood Suites (5.7 miles from IAS) 3819 US Highway 1, Princeton, NJ 08540 - 609-720-0550
  • DoubleTree (5.9 miles from IAS) 4355 US Highway 1, Princeton, NJ 08540 - 609-452-2400

Please inquire with the hotel if they offer shuttle service to the local Princeton area.

events: