How to Start Learning Deep Learning

This post was written by Ofir Press. Ofir is a graduate student at Tel-Aviv University’s Deep Learning Lab. His main focus is on using deep learning for natural language processing.

"Due to the recent achievements of artificial neural networks across many different tasks (such as face recognition, object detection and Go), deep learning has become extremely popular. This post aims to be a starting point for those interested in learning more about it.

If you already have a basic understanding of linear algebra, calculus, probability and programming: I recommend starting with Stanford’s CS231n. The course notes are comprehensive and well-written. The slides for each lesson are also available, and even though the accompanying videos were removed from the official site, re-uploads are quite easy to find online.

If you don’t have the relevant math background: There is an incredible amount of free material online that can be used to learn the required math knowledge. Gilbert Strang’s course on linear algebra is a great introduction to the field. For the other subjects, edX has courses from MIT on both calculus and probability.

If you are interested in learning more about machine learning: Andrew Ng’s Coursera class is a popular choice as a first class in machine learning. There are other great options available such as Yaser Abu-Mostafa’s machine learning course which focuses much more on theory than the Coursera class but it is still relevant for beginners. Knowledge in machine learning isn’t really a prerequisite to learning deep learning, but it does help. In addition, learning classical machine learning and not only deep learning is important because it provides a theoretical background and because deep learning isn’t always the correct solution.

CS231n isn’t the only deep learning course available online. Geoffrey Hinton’s Coursera class “Neural Networks for Machine Learn... covers a lot of different topics, and so does Hugo Larochelle’s “Neural Networks Class”. Both of these classes contain video lectures. Nando de Freitas also has a course available online which contains videos, slides and also a list of homework assignments.

If you prefer reading over watching video lectures: Neural Networks and Deep Learning is a free online book for beginners to the field. The Deep Learning Book is also a great free book, but it is slightly more advanced.

Where to go after you’ve got the basics:

Computer Vision is covered by most, if not all, of the deep learning resources mentoined above.
Recurrent Neural Networks (RNNs) are the basis of neural network based models that solve tasks related to sequences such as machine translation or speech recognition. Andrej Karpathy’s blog post on RNNs is a great place to start learning about them. Christopher Olah has a great blog where many deep learning concepts are explained in a very visual and easy to understand way. His post on LSTM networks is an introduction to LSTM networks which are a wildly used RNN variant.
Natural Language Processing (NLP): CS224d is an introduction to NLP with deep learning. Advanced courses are available from both Kyunghyun Cho (with lecture notes here) and Yoav Goldberg.
Reinforcement Learning: If you’d like to control robots or beat the human champion of Go, you should probably use reinforcement learning. Andrej Karpathy’s post on deep reinforcement learning is an excellent starting point. David Silver also recently published a short blog post introducing deep reinforcement learning.

Deep learning frameworks: There are many frameworks for deep learning but the top three are probably Tensorflow (by Google), Torch (by Facebook) and Theano (by MILA). All of them are great, but if I had to select just one to recommend I’d say that Tensorflow is the best for beginners, mostly because of the great tutorials avialable.

If you’d like to train neural networks you should probably do it on a GPU. You dont have to, but its much faster if you do. NVIDIA cards are the industry standard, and while most research labs use $1000 dollar graphics cards, there are a few affordable cards that can also get the work done. An even cheaper option is to rent a GPU-enabled instance from a cloud server provider like Amazon’s EC2 (short guide here).

Good luck!"

Find the original article, here.

DSC Resources

Career: Training | Books | Cheat Sheet | Apprenticeship | Certification | Salary Surveys | Jobs
Knowledge: Research | Competitions | Webinars | Our Book | Members Only | Search DSC
Buzz: Business News | Announcements | Events | RSS Feeds
Misc: Top Links | Code Snippets | External Resources | Best Blogs | Subscribe | For Bloggers

Additional Reading

8 members like this

Comment

You need to be a member of Data Science Central to add comments!

Join Data Science Central

Comment by Sione Palu on September 12, 2016 at 12:35am: In my opinion, deep-learning is just a tiny fraction of toolsets available that are available today for doing analytics. I just can't think of how I would use it apart from doing image classification since analytics is a huge area where certain tasks suit certain techniques & tools. Deep learning is still a young domain which it hasn't been applied outside to other non-traditional machine learning field where certain existing techniques are well catered for. For example is the system-identification domain. Deep learning hasn't appeared in that domain, it will in the future (no doubt), but currently the existing shallow neural network techniques are still dominant in non-linear-system-identification development of today. The Matlab System Identification toolbox still have shallow neural network for system identification on non-linear system & their latest version of the toolbox has added a hybrid of wavelet-neural-network functionality to it, ie, the neural network activation function is wavelet-based.