Instead of seeing each Machine Learning (ML) method as a “shiny new object”, here is an attempt to create a unified picture. There is no consensus when it comes to an ontology for ML methods; organizational principles are simply ways to get our arms around knowledge so that we are not swamped by too many unconnected notions.


A powerful organization of the concepts or Ontology of ML is based on conditional expectation.


Conditional Expectation of Class ‘y’ given input attributes, x, denoted by E[y | x].

Implementation of estimation of the conditional expectation with various assumptions lead, one way or the other, to ALL the ML techniques that we have today in 2016.

In chapter 4 (“Modern” ML Method) of my upcoming book, “SYSTEMS Analytics”, we develop the basic theory and algorithms for some key blocks in the diagram above. State-space method is the subject matter of Part II –Systems Analytics, the future evolution” – of the book.

ML is a practical pursuit! In ML practice, these ML methods are “wrapped” by “bootstrap” and “consensus” methods.


Input side: Bootstrap methods

The objective is to maximize Training Set information use.

  • There are many approaches:
    • Bootstrap training set; sample Training Set with replacement.
    • Monte Carlo methods; for probability density estimation.
    • Feature subspace.
    • Design of Experiments principles from Statistics; “blocking”, factorial design, etc.
    • And many more.


Output side: Consensus methods

Solve the problem using independent ML methods and combine the results.

  • Combine “weak learners”.
    • Random Forest.
    • AdaBoost.
    • And many more.


By the way, here are the Chapter 4 topics.

If you are interested in reading this chapter, please connect on LinkedIn and email me for a pre-publication copy. I will be happy to send one for your personal use.



[email protected]


About the Author:

Dr. PG Madhavan is the Founder of Syzen Analytics, Inc.  He developed his expertise in Analytics as an EECS Professor, Computational Neuroscience researcher, Bell Labs MTS, Microsoft Architect and startup CEO.  PG has been involved in four startups with two as Founder.

More at www.linkedin.com/in/pgmad


Views: 9608


You need to be a member of Data Science Central to add comments!

Join Data Science Central

Comment by Sung-Kook Han on November 7, 2017 at 4:47pm

Good ~~~

© 2021   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service