Hi all,

I'm writing to share a book I just published that I think many of you might find interesting or useful.

The book is called "Machine Learning from Scratch." It provides complete derivations of the most common algorithms in ML (OLS, logistic regression, naive Bayes, trees, boosting, neural nets, etc.) both in theory and math. It also demonstrates constructions of each of these methods from scratch in Python using only numpy.

My aim with the book is to provide a very thorough rundown of the fitting process behind the algorithms we see every day. I hope that seeing the models derived in math or constructed in code helps readers understand the models at a deeper level and feel more comfortable optimizing them for their own work.

Any comments or questions would be very much appreciated either on this post, on the book's github, or to me directly at [email protected]

*The book is available here*

This book covers the building blocks of the most common methods in machine learning. This set of methods is like a toolbox for machine learning engineers. Those entering the field of machine learning should feel comfortable with this toolbox so they have the right tool for a variety of tasks. Each chapter in this book corresponds to a single machine learning method or group of methods. In other words, each chapter focuses on a single tool within the ML toolbox.

In my experience, the best way to become comfortable with these methods is to see them derived from scratch, both in theory and in code. The purpose of this book is to provide those derivations. Each chapter is broken into three sections. The *concept* sections introduce the methods conceptually and derive their results mathematically. The *construction* sections show how to construct the methods from scratch using Python. The *implementation* sections demonstrate how to apply the methods using packages in Python like `scikit-learn`

, `statsmodels`

, and `tensorflow`

.

There are many great books on machine learning written by more knowledgeable authors and covering a broader range of topics. In particular, I would suggest An Introduction to Statistical Learning, Elements of Statistical Learning, and Pattern Recognition and Machine Learning, all of which are available online for free.

While those books provide a conceptual overview of machine learning and the theory behind its methods, this book focuses on the bare bones of machine learning algorithms. Its main purpose is to provide readers with the ability to construct these algorithms independently. Continuing the toolbox analogy, this book is intended as a user guide: it is not designed to teach users broad practices of the field but rather how each tool works at a micro-level.

This book is for readers looking to learn new machine learning algorithms or understand algorithms at a deeper level. Specifically, it is intended for readers interested in seeing machine learning algorithms derived from start to finish. Seeing these derivations might help a reader previously unfamiliar with common algorithms understand how they work intuitively. Or, seeing these derivations might help a reader experienced in modeling understand how different algorithms create the models they do and the advantages and disadvantages of each one.

This book will be most helpful for those with practice in basic modeling. It does not review best practices—such as feature engineering or balancing response variables—or discuss in depth when certain models are more appropriate than others. Instead, it focuses on the elements of those models.

The *concept* sections of this book primarily require knowledge of calculus, though some require an understanding of probability (think maximum likelihood and Bayes’ Rule) and basic linear algebra (think matrix operations and dot products). The appendix reviews the math and probability needed to understand this book. The concept sections also reference a few common machine learning methods, which are introduced in the appendix as well. The concept sections do not require any knowledge of programming.

The *construction* and *code* sections of this book use some basic Python. The construction sections require an understanding of the corresponding content sections and familiarity creating functions and classes in Python. The code sections require neither.

- Juniper adds Mist AIOps to its 128 Technology-based SD-WAN
- 10 microservices patterns all architects should know
- IBM extends Call for Code for Racial Justice program
- citizen development
- How to manage third-party risk in the supply chain
- Gartner predicts data storytelling will dominate BI by 2025
- AWS Data Exchange and the third-party cloud data marketplace
- Overcome common IoT edge computing architecture issues

Posted 1 March 2021

© 2021 TechTarget, Inc. Powered by

Badges | Report an Issue | Privacy Policy | Terms of Service

**Most Popular Content on DSC**

To not miss this type of content in the future, subscribe to our newsletter.

- Book: Applied Stochastic Processes
- Long-range Correlations in Time Series: Modeling, Testing, Case Study
- How to Automatically Determine the Number of Clusters in your Data
- New Machine Learning Cheat Sheet | Old one
- Confidence Intervals Without Pain - With Resampling
- Advanced Machine Learning with Basic Excel
- New Perspectives on Statistical Distributions and Deep Learning
- Fascinating New Results in the Theory of Randomness
- Fast Combinatorial Feature Selection

**Other popular resources**

- Comprehensive Repository of Data Science and ML Resources
- Statistical Concepts Explained in Simple English
- Machine Learning Concepts Explained in One Picture
- 100 Data Science Interview Questions and Answers
- Cheat Sheets | Curated Articles | Search | Jobs | Courses
- Post a Blog | Forum Questions | Books | Salaries | News

**Archives:** 2008-2014 |
2015-2016 |
2017-2019 |
Book 1 |
Book 2 |
More

**Most popular articles**

- Free Book and Resources for DSC Members
- New Perspectives on Statistical Distributions and Deep Learning
- Time series, Growth Modeling and Data Science Wizardy
- Statistical Concepts Explained in Simple English
- Machine Learning Concepts Explained in One Picture
- Comprehensive Repository of Data Science and ML Resources
- Advanced Machine Learning with Basic Excel
- Difference between ML, Data Science, AI, Deep Learning, and Statistics
- Selected Business Analytics, Data Science and ML articles
- How to Automatically Determine the Number of Clusters in your Data
- Fascinating New Results in the Theory of Randomness
- Hire a Data Scientist | Search DSC | Find a Job
- Post a Blog | Forum Questions

## You need to be a member of Data Science Central to add comments!

Join Data Science Central