Instead of seeing each Machine Learning (ML) method as a “shiny new object”, here is an attempt to create a unified picture. There is no consensus when it comes to an ontology for ML methods; organizational principles are simply ways to get our arms around knowledge so that we are not swamped by too many unconnected notions.
A powerful organization of the concepts or Ontology of ML is based on conditional expectation.
Conditional Expectation of Class ‘y’ given input attributes, x, denoted by E[y | x].
Implementation of estimation of the conditional expectation with various assumptions lead, one way or the other, to ALL the ML techniques that we have today in 2016.
In chapter 4 (“Modern” ML Method) of my upcoming book, “SYSTEMS Analytics”, we develop the basic theory and algorithms for some key blocks in the diagram above. State-space method is the subject matter of Part II – “Systems Analytics, the future evolution” – of the book.
ML is a practical pursuit! In ML practice, these ML methods are “wrapped” by “bootstrap” and “consensus” methods.
Input side: Bootstrap methods
The objective is to maximize Training Set information use.
Output side: Consensus methods
Solve the problem using independent ML methods and combine the results.
By the way, here are the Chapter 4 topics.
If you are interested in reading this chapter, please connect on LinkedIn and email me for a pre-publication copy. I will be happy to send one for your personal use.
About the Author:
Dr. PG Madhavan is the Founder of Syzen Analytics, Inc. He developed his expertise in Analytics as an EECS Professor, Computational Neuroscience researcher, Bell Labs MTS, Microsoft Architect and startup CEO. PG has been involved in four startups with two as Founder.
More at www.linkedin.com/in/pgmad