Additional Sources

Textbooks

1. Machine Learning: A probabilistic perspective by Kevin Murphy. The material in this book is closest to what we will cover in the course, but is unfortunately not available for free. Written by an academic and a practitioner of machine learning, this text is full of real world examples and applications, while eschewing the ad-hoc approach of other texts.
2. Introduction to statistical learning by Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani. This textbook is available for free. It has minimal prerequisites and as a consequence does not touch on some topics of interest.  On the other hand, it has detailed R exercises which are wonderful.
3. The elements of statistical learning by Trevor Hastie, Robert Tibshirani, and Jerome Friedman. A more advanced version of the previous text (and also free), covering more material, and assuming more background in statistics.
4. Deep learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville. A new text on this burgeoning field. Written by central figures in the area, this text surveys deep learning, includes many useful citations (a great deal of the literature is still in academic articles), provides good rules of thumb, and is a relatively light read. This book is very much from a practitioner’s perspective with less justification for methodology and less emphasis on mathematical rigor. My limited experience is that other papers in this area have a similar emphasis, which reflects the limited understanding of why these methods work very well in some instances and not so well in others.
5. Reinforcement Learning: An Introduction by Richard Sutton and Andrew Barto. A free textbook that has been recently revised and covers quite a few topics.

Lecture series

1. Trevor Hastie and Robert Tibshirani have made their lectures and slides for their course on statistical learning available. These are very well done.
2. Machine Learning lectures from Georgia Tech, these are very well done introductory lectures on machine learning directed toward engineers. I personally love the nerdy jokes interspersed throughout.
3. Deep learning course lectures by Nando de Freitas. Excellent lecture series on deep learning from Oxford for computer scientists. This lecture series is aimed at those with some more background.
4. Machine learning course lectures by Andrew Ng. A somewhat older lecture series from Stanford aimed at computer scientists.
5. Machine Learning 101 by Jason Mayes. A very well done overview of the subject.

Websites

1. Datacamp: An online learning site with an emphasis on data science. Includes mini-lectures and online programming exercises. A nice introduction on how to take the material we are learning and apply it. However, there is a noticeable bump in difficulty in proceeding to implement these methods from scratch on your own computer.
2. Andrej Karpathy’s blog: A truly excellent sequence of blog posts on topics related to machine learning.
3. Kaggle: Hosts machine learning competitions, allows machine learning practitioners to share information, and hosts many large datasets.
4. Seeing Theory: A beautiful sequence of javascript visualizations for statistical concepts.

Programming references

1. Introduction to R.
2. Python 3 standard library.
3. Keras (a significantly simpler way to apply the Tensorflow (or another backend) library for building neural networks).
4. Tensorflow API (a rich library for building and using neural networks).