I originally started writing this notebook to serve as an introduction decision trees. It’s a description of the “1-rule” algorithm which I think is worth studying for the following reasons:
- It’s arguably the simplest and most useful machine learning algorithm you can learn
- It’s a simple introduction to “decision trees”
- It’s a simple introduction to “information entropy“
- It has minimal mathematical content, so that anyone technical can follow the ideas
I first encountered the “1 Rule” prediction algorithm in ‘Data Mining Practical Machine Learning Tools and Techniques’ by ‘Ian H. Witton’. A book which serves as a great introduction to machine learning ideas.
Content of the Notebook
- If you had to choose just one feature?
- If you didn’t have any features?
- Information Entropy
- Reducing Disorder
- Feature Selection
- Going Further
You can access the notebook here.
