Deep Learning

NIPS December 5-10, 2016 Barcelona: Registration:

OpenAi’s Machine Learning Unconference: October 7-8, 2016, SanFrancisco .. Apply here:

Deep Learning School (!)   September 24-25, 2016 … Apply here:

First Workshop on Representation Learning in NLP – Berlin – August 11, 2016 –

Computer Vision – CVPR 2016 – June 26 – July 1, 2016

ICML 2016 – New York – June 19-24, 2016

NAACL 2016 – San Diego – June 12-17, 2016

MLSS-2016 – Reinforcement Learning – John Schulman – Deep Reinforcement Learning (Lecture 1, Lecture 2, Lecture 3, Lecture 4)  … the full schedule of Machine Learning Summer School 2016 – Cadiz (May 11-21, 2016)  is here.  The Machine Learning Summer School 2016 – Peru (Aug 2-13, 2016) is here.

ICLR 2016:  May 2-4, Schedule:  and News:

Richard Socher’s Course (March-June 2016) at Stanford: CS224d: Deep Learning for Natural Language Processing (videos here )

Andrej Karpathy’s Course (Jan-March 2016) at Stanford: CS231n: Convolutional Neural Nets for Visual Recognition (videos here as well: here)

A set of talks posted by Deep Hack QA – (many are in Russian, but here are the ones in English (slides here), the original website is:

  • Kyunghyun Cho – New Territory of Machine Translation, here
  • Christopher Manning – Compositional Linguistic Représentations, here
  • Konstantin Vorontsov – Regularization of Topic Models for Question Answering, here
  • Phil Blunsom – Memory, Reading, and Comprehension, here
  • Jason Weston – Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems, here
  • Anatoly Levenchuk – Machine learning engineering, here
  • Wojciech Zaremba – Learning Simple Algorithms from Examples, here
  • Tomas Mikolov – The Roadmap towards Machine Intelligence, here
  • Quoc Le – Deep Learning for Language Understanding, here
  • Oriol Vinyals – Sequences in Deep Learning, here
  • Jonathan Berant – Mapping natural language utterances to logical form, here
  • Rob Fergus – The MazeBase project, here
  • Dmitry Vetrov – Breaking Sticks and Ambiguities with Adaptive Skip-gram, here

Kyunghyun Cho’s course at NYU – Natural Language Understanding with Distributed Representations, Fall 2015 videos here.

Video Recordings of ICML 2015 – Deep Learning Workshop – here related youtube: here

Richard Socher’s Course (Mar-June 2015) at Stanford: CS224d: Deep Learning for Natural Language Processing

Udacity’s Course on Deep Learning announced Jan 21, 2016 : Deep Learning at Udacity taught by Google

NIPS 2015:  jam packed with deep learning talks and workshops: NIPS 2015 took place in Montreal, December 7-12, 2015: Tutorial videos here:

Nvidia has a set of resources here:

Deep Learning Summer School: Montreal August 3-12, 2015: Application Deadline March 15, 150 students only: details here. Organizers: Yann LeCun and Yoshua Bengio.

Yann LeCun’s Course at NYU: 2012-2016:

A talk at MIT by Geoff Hinton (Google, UofT), December 4, 2014 – Fall Colloquium Series … What is wrong with ‘standard’ neural nets?

A wonderful interview and overview of deep learning as it relates to images and text … at Data Driven Meetup’s December 2014 with Yann LeCun : here. He mentions that he will be teaching a course in Deep Learning at NYU Spring 2015 and those lectures will be available online. Here is the link to the videos of Yann LeCun’s 2014 Deep Learning course at NYU

A very accessible course at Coursera on Neural Networks for Machine Learning by Geoffrey Hinton (October 2012): he imparts tons of wonderful intuition. Highly recommended. It doesn’t include recent work on NLP, but just well worth your time.

Recent open-sourced software:  Facebook and Torch, Google’sTensor Flow

Blogs:  by Ilya Sutskever (guest blogger). and another good one by Radim Rehurek on NLP and word2vec.

Deep Learning at NIPS 2014:

  • cuDNN: Efficient Primitives for Deep Learning: Chetlur, Woolley, Vandermersch, Cohen, Tran (Nvidia), Catanzaro (Baidu), Shelhamer (Berkeley)  : here
  • Distilling the Knowledge in a Neural Network: Geoffrey Hinton, Oriol Vinyals, Jeff Dean (Google): here
  • Supervised Learning in Dynamic Bayesian Netowrks : Shamim Nemati, Ryan Adams (Harvard) : here
  • Deeply Supervised Nets: Lee, Xie, Gallagher, Zhang, Tu (UCSD, Microsoft) : here
  • Posters: here

Deep Learning Master Class in Tel Aviv, November 5-6, 2014, here.

  • Yann LeCun (Facebook) Unreasonable Effectiveness of Deep Learning
  • Yaniv Taigman (Facebook) Web-Scale Training for Face Identification
  • Amnon Shashua (Hebrew U) SimNets – A Generalization of Convolutional Networks
  • Rob Fergus (Facebook) Learning to Discover Efficient Mathematical Identities
  • Shai Shalev Shwartz (Hebrew U) – Accelerating Stochastic Optimization
  • Ilya Sutskever (Google) Supervised Learning with Deep Neural Networks
  • Nati Srebro (Technion) Approximation, Generalization and Computation – Neural Networks as Universal Learners
  • Yoshua Bengio (University of Montreal) Fundamentals of Deep Learning of Representations


Several interesting talks at Gigaom’s Future of AI Conference.

An intro to Deep Learning by Yoshua Bengio

A fantastic talk by Yann LeCunThe Unreasonable Effectiveness of Deep Learning” covering convolutional neural nets from the beginning ….

A book by Microsoft researchers: Li Deng and Dong Yu, Deep Learning: Methods and Applications, published January 2014.

  • Deep Learning: Methods and Applications, originally published as Foundations and Trends in Signal Processing.  The monograph contains a list of conferences and tutorials on deep learning and its applications to signal and information processing from 2008-2013. 

Screen Shot 2014-09-27 at 11.03.13 PM Screen Shot 2014-09-27 at 11.03.20 PM

Oren Etzioni on Reddit (yes, Reddit):   here. Some quotes: 

  • Q: What use do ontologies have in knowledge representation for reasoning systems? Or is it just better to only use open-domain representations?
  • A: Ontologies are straight jackets, often, so I prefer open systems BUT open systems can have lower precision. We are moving towards hybrid systems that leverage ontologies but have graceful decay when the ontologies fail or are stale.
  • Q: what are your thoughts on Semantic Web which allows for knowledge representation, reasoning and has backing of TBL? Do you see it going as mainstream as ML/while leveraging success of ML?
  • A: The semantic web has not proven to be a practical framework, but linked open data, which has grown out of TBL’s ideas, is very exciting. I see that as becoming increasingly mainstream.

A not very ordered but interesting Annotated Deep Learning Bibliography

  • 3D, Algorithm, Applications, Architecture, Asynchronous, Audio, Autoencoder, Big Data, Bioinformatics, Brain, BrainWaves, Challenges, Convolutional Network, Deep Belief Network, EEG, Emotion, Emotion Detection, Energy Efficiency, Face Detection, Face Recognition, Feature Extraction, Finance, Games GPU, Hardware, Healthcare, Image Recognition, Information Retrieval, Infrastructure, Kernel Methods, MachineTranslation, Medicine, Memristor, Mine Detection, Mobile, Motion Detection, Multicore, NLP, Network, Neuromorphic, Noise, Noisy Data, Overview, Parallelization, Part-of-Speech, Performance Improvement, Physics, Platform, Recommender Systems, Regularization, Reinforcement Learning, Retricted Boltzmann Machines, Robotics, Search, Sentiment Analysis, Simulation, Sparseness, Speech Recognition, Stochastic Gradient, Stochastic Gradient Descent, Survey, Time Series, Video, Voice Recognition

Some people/companies working in the area

VLAB conference, Deep Learning: Intelligence from Big Data (September 22, 2014, at Stanford):

  • Steve Jurvetson DFJ Ventures,  @dfjsteve, moderator, and introduction
  • Naveen Rao, at Nervana Systems @nervanasys (starts at 40:20, hardware)
  • Ilya Sutskever, Google Brain, at Google+(starts at 43:23min)
  • Elliot Turner, AlchemyAPI, @eturner303 (starts at 46:00min)
  • Adam Berenzweig, Clarifai, @madadam (at 22:40min, object/img recog)


Hugo Larochelle – Neural Networks Class – Universite de Shelbrooke – Fall 2013  – web page here and videos here.

NIPS2009 – Deep Learning in Natural Language Processing – Jason Weston and Ronan Collobert

© 2014-2015 ContextBridge, Inc.

Leave a Reply

Your email address will not be published. Required fields are marked *