This was a great question posted on Quora.com, and attracted many comments. Here we summarize the must interesting contributions for you.

Source for picture: click here

Books

• Hastie, Tibshirani, and Friedman's The Elements of Statistical Learning
• Bishop's Pattern Recognition and Machine Learning
• David Barber's Bayesian Reasoning and Machine Learning
• Kevin Murphy's Machine learning: a Probabilistic Perspective

Advice

• See this talk by Jeremy Howard : At Kaggle, It’s a Disadvantage To Know Too Much and “Getting In Shape For The Sport Of Data Science”
• Start by practicing on toy datasets in MATLAB and walking through simple examples in Statistics and Neural Network Toolbox
• Below two books are standard introduction texts. They are complementary,   the first one is written from a statistician perspective with lots of data analysis examples and the second one is focusing on algorithms. Both are graduate level texts requiring knowledge of algebra, statistics and calculus. It would also help to take a  class on optimization but not strictly necessary. 

• For a more gentle read you may want to check out the textbook by Mitchell (edit:  and his lectures, thanks to Willy Kuo)
• The most coherent and concise lecture notes are by Ng, Also see his course.

Some applications and advanced topics

• Language analysis course and notes
• Deep learning: ANN + UFLDL + CNN + ELM tutorials
• Robotics and Game AI
• Cognition and Neural models
• Lifelong learning

Training

• Start with the new ISL book "An Introduction to Statical Learning"http://www-bcf.usc.edu/~gareth/ISL/
• Very approachable.
• Do the hands-on lab at the end of each chapter.
• Michael has done a good job in listing the most relevant materials. But the order is important. MMonk's materials are very good, but dense.
• ISL Class Stanford/Hastie/Tibsharani at their best - Statistical Learning
• Prof. Pedro Domingo, coursera.org, Coursera
• Or Prof. Andrew Ng, Coursera
• Prof. Abu Mostafa, CaltechX: CS1156x: Learning From Data
• Then mathematicalmonk's channel

And of course, check out our machine learning resources on DSC.