]]>

]]>

DML stands for “Dynamical Machine Learning” (more in the book, “SYSTEMS Analytics for IoT Data Science”, 2017). This match is not surprising once you realize that DML & IoT are both based on the venerable Systems Theory. Let us dig deeper . . .Consider IoT for industrial applications. A machine is instrumented with sensors, data are collected in real-time (or at intervals), communicated to the cloud where IoT Data Science techniques predict machine condition which results in an action, if necessary, such as repair action on the machine. This is a classic “closed-loop” system. The theory that abstracts and governs this closed-loop system is the subject matter of Systems Theory, an undergraduate engineering topic.Systems Theory is broad and deep – in the past 70 or so years, a great body of work has been developed from deep theory to day-to-day applications such as GPS in your mobile phones, controlling massive chemical plants or Dreamliner airplanes. Systems Theory’s state-space model based methods allow you to describe, estimate/predict and control all parts of a closed-loop system.DML is a topic in “Systems Analytics” (“SYSTEMS Analytics for IoT Data Science”, 2017). A key algorithm to implement DML is called “Rocket” Kalman algorithm developed in the book. As opposed to “static” ML (a more illuminating operational definition is “learn-once-and-use-for-ever” method!), “dynamical” ML permits continuous learning. For long term use of IoT for machinery monitoring and for rapidly changing systems, it is obvious that *dynamic* or *continuous* learning will be much more appropriate and hence more accurate, robust and reliable. State-space:In general terms, state space is the “space” in which the machine “exists”. Of course, quantitative aspects of the machine alone are captured in this “space”. It could be graphical (state-space trajectories, for example) or a list of the value of the “states” for each time interval.Closed-loop system’s evolution over time is fully captured in these diagrams. While the visual representation by itself can be revealing (in some cases), we use the “state” equations in which these values are embedded as parameters for quantitative operations such as estimation and prediction. Since data associated with systems such as machine vibration or temperature are generally random, statistical methods are employed to obtain useful predictions such as, “Where is the State trajectory going next?”. This is the right system-theoretic question to ask of your machine’s future condition! Here is a picture of “Rocket” Kalman for DML. State Space Model:s[n] = A s[n-1] + q[n-1]y[n] = H[n] s[n] + r[n] s[n] in the equations are the “States” that we have been discussing. As you can see, if we knew the States, s[n], and a few other quantities, we can calculate the output, y[n]. In certain cases, this can be formulated as the *prediction of machine condition* that we are interested in!In summary, the “Rocket” Kalman block diagram is a general “Digital Twin” of our machine and the values of the States fully quantify a *specific* machine – ‘Digital Twin of Machine serial number: xxx’. DML for IoT Machine Learning:From the foregoing discussion, you got a glimpse of the basis for the assertion, “DML & IoT are based on Systems Theory”. A less technical discussion follows. If we took a still picture of an athlete competing in a hurdles race, the picture of the runner will be fuzzy due to her movement. One the other hand, if we had a video camera, we can record the race faithfully. The current “static” machine learning (which I call “Loue” for Learn Once & Use for Ever!) is akin to the still camera picture and DML is like a video!It is also notable that each video *frame* will have captured the runner in action at a particular instant faithfully (because of the frequent “updates” of the picture) without the fuzziness! This is a window into how to use DML for cases where “static” ML may suffice from an application/business value point of view. Observe that the video frame picture is NOT fuzzy but the still camera picture is; the “clarity” provided by the video frame will give us better results when DML is used for “LOUE” applications instead of Static ML. What does this mean in practice?In “ML speak”, DML is used for “learning” from the Training Set. NOTE that DML is the canonical solution for CONTINUOUS machine learning; “Rocket” Kalman is an algorithm to realize DML. What is learned is the State “evolution” – a “video” of States in the first picture – what we call a “Digital Twin Video”.For each Feature Vector in the TEST Set, we find the corresponding “video frame” or the “vector of State values” (using some similarity measure) that will provide the best estimate of the ML output. The theory described in my book, “SYSTEMS Analytics for IoT Data Science”, shows that this output is the OPTIMAL estimate in the Bayesian sense. This is the best we can do! DML is a powerful framework based on Systems Theory which also underpins IoT closed-loop systems. “Rocket” Kalman is just one example of an algorithm, but optimal in the Bayesian sense. “Systems” thinking and new algorithms can be built up on this DML framework for diverse IoT applications. PG Madhavan, Ph.D. - “LEADER . . . of a life in pursuit of excellence . . . in IoT Data Science”https://www.linkedin.com/in/pgmad See More

DML stands for “Dynamical Machine Learning” (more in the book, “SYSTEMS Analytics for IoT Data Science”, 2017). This match is not surprising once you realize that DML & IoT are both based on the venerable Systems Theory. Let us dig deeper . . .Consider IoT for industrial applications. A machine is instrumented with sensors, data are collected in real-time (or at intervals), communicated to the cloud where IoT Data Science techniques predict machine condition which results in an action, if necessary, such as repair action on the machine. This is a classic “closed-loop” system. The theory that abstracts and governs this closed-loop system is the subject matter of Systems Theory, an undergraduate engineering topic.Systems Theory is broad and deep – in the past 70 or so years, a great body of work has been developed from deep theory to day-to-day applications such as GPS in your mobile phones, controlling massive chemical plants or Dreamliner airplanes. Systems Theory’s state-space model based methods allow you to describe, estimate/predict and control all parts of a closed-loop system.DML is a topic in “Systems Analytics” (“SYSTEMS Analytics for IoT Data Science”, 2017). A key algorithm to implement DML is called “Rocket” Kalman algorithm developed in the book. As opposed to “static” ML (a more illuminating operational definition is “learn-once-and-use-for-ever” method!), “dynamical” ML permits continuous learning. For long term use of IoT for machinery monitoring and for rapidly changing systems, it is obvious that *dynamic* or *continuous* learning will be much more appropriate and hence more accurate, robust and reliable. State-space:In general terms, state space is the “space” in which the machine “exists”. Of course, quantitative aspects of the machine alone are captured in this “space”. It could be graphical (state-space trajectories, for example) or a list of the value of the “states” for each time interval.Closed-loop system’s evolution over time is fully captured in these diagrams. While the visual representation by itself can be revealing (in some cases), we use the “state” equations in which these values are embedded as parameters for quantitative operations such as estimation and prediction. Since data associated with systems such as machine vibration or temperature are generally random, statistical methods are employed to obtain useful predictions such as, “Where is the State trajectory going next?”. This is the right system-theoretic question to ask of your machine’s future condition! Here is a picture of “Rocket” Kalman for DML. State Space Model:s[n] = A s[n-1] + q[n-1]y[n] = H[n] s[n] + r[n] s[n] in the equations are the “States” that we have been discussing. As you can see, if we knew the States, s[n], and a few other quantities, we can calculate the output, y[n]. In certain cases, this can be formulated as the *prediction of machine condition* that we are interested in!In summary, the “Rocket” Kalman block diagram is a general “Digital Twin” of our machine and the values of the States fully quantify a *specific* machine – ‘Digital Twin of Machine serial number: xxx’. DML for IoT Machine Learning:From the foregoing discussion, you got a glimpse of the basis for the assertion, “DML & IoT are based on Systems Theory”. A less technical discussion follows. If we took a still picture of an athlete competing in a hurdles race, the picture of the runner will be fuzzy due to her movement. One the other hand, if we had a video camera, we can record the race faithfully. The current “static” machine learning (which I call “Loue” for Learn Once & Use for Ever!) is akin to the still camera picture and DML is like a video!It is also notable that each video *frame* will have captured the runner in action at a particular instant faithfully (because of the frequent “updates” of the picture) without the fuzziness! This is a window into how to use DML for cases where “static” ML may suffice from an application/business value point of view. Observe that the video frame picture is NOT fuzzy but the still camera picture is; the “clarity” provided by the video frame will give us better results when DML is used for “LOUE” applications instead of Static ML. What does this mean in practice?In “ML speak”, DML is used for “learning” from the Training Set. NOTE that DML is the canonical solution for CONTINUOUS machine learning; “Rocket” Kalman is an algorithm to realize DML. What is learned is the State “evolution” – a “video” of States in the first picture – what we call a “Digital Twin Video”.For each Feature Vector in the TEST Set, we find the corresponding “video frame” or the “vector of State values” (using some similarity measure) that will provide the best estimate of the ML output. The theory described in my book, “SYSTEMS Analytics for IoT Data Science”, shows that this output is the OPTIMAL estimate in the Bayesian sense. This is the best we can do! DML is a powerful framework based on Systems Theory which also underpins IoT closed-loop systems. “Rocket” Kalman is just one example of an algorithm, but optimal in the Bayesian sense. “Systems” thinking and new algorithms can be built up on this DML framework for diverse IoT applications. PG Madhavan, Ph.D. - “LEADER . . . of a life in pursuit of excellence . . . in IoT Data Science”https://www.linkedin.com/in/pgmad See More

]]>

]]>

]]>

]]>

Internet of Things (IoT) has generated a ton of excitement and furious activity. However, I sense some discomfort and even dread in the IoT ecosystem about the future – typical when a field is not growing at a hockey-stick pace . . .“History may not repeat itself but it rhymes”, Mark Twain may have said. What history does IoT rhyme with? I have often used this diagram to crisply define IoT.Even 10 years ago, the first two blocks in the diagram were major challenges; in 2017, sensors, connectivity, cloud and Big Data are entirely manageable. But extracting insights and more importantly, applying the insights in, say an industrial environment, is still a challenge. While there are examples of business value generated by IoT, the larger value proposition beyond these islands of successes is still speculative. How do you make it real in the fastest possible manner?In a slogan form, the value proposition of IoT is ”Do more at higher quality with better user experience”. Let us consider a generic application scenario in industrial IoT.IoT Data Science prescribes actions (“prescriptive analytics”) which are implemented, outcomes of which are monitored and improved over time. Today, humans are involved in this chain, either as observers or as actors (picking a tool from the shelf and attaching it to the machine).BTW, when I mentioned “Better UX” in the slogan, I was referring to this human interaction elements improved by “Artificial Intelligence” via natural language or visual processing.Today and for the foreseeable future, IoT Data Science is achieved through Machine Learning which I think of as “competence without comprehension” (Dennett, 2017). We cannot even agree on what human intelligence or comprehension is and I want to distance myself from such speculative (but entertaining) parlor games!Given such a description of the state of IoT art in 2017, it appears to me that what is preventing us from hockey-stick growth is the state of IoT Data Science. The output of IoT Data Science has to serve two purposes: (1) insights for the humans in the loop and (2) lead us to closed-loop automation, BOTH with the business objective of “Do More at Higher Quality” (or increased throughput and continuous improvement).Machine Learning has to evolve and evolve quickly to meet these two purposes. One, IoT Data Science has to be more “democratized” so that it is easy to deploy for the humans in the loop – this work is underway by many startups and some larger incumbents. Two, Machine Learning has to become *continuous* learning for continuous improvement which is also at hand (NEXT Machine Learning Paradigm: “DYNAMICAL" ML).With IoT defined as above, when it comes to “rhyming with history”, I make the point (in Neural Plasticity & Machine Learning blog) that the current Machine Learning revolution is NOT like the Industrial Revolution (of steam engine and electrical machines) which caused productivity to soar between 1920 and 1970; it is more like the Printing Press revolution of the 1400s!Printing press and movable type played a key role in the development of Renaissance, Reformation and the Age of Enlightenment. Printing press created a disruptive change in “information spread” via augmentation of “memory”. Oral tradition depended on how much one can hold in one’s memory; on the printed page, memories last forever (well, almost) and travel anywhere.Similarly, IoT Data Science is in the early stages of creating disruptive change in “competence spread” via Machine Learning (which is *competence without comprehension*) based on Big Data analysis. Humans can process only a very limited portion of Big Data in their heads; Data Science can make sense of Big Data and provide competence for skilled actions. To make the correspondence explicit, "information spread" in the present case is "competence spread"; "memory" analog is "learning" and "printed page" is "machine learning". Just like Information Spread was enhanced by “augmented memory” (via printed page), Competence Spread will be enhanced by Machine Learning. Information Spread and the Printing Press “revolution” resulted in Michelangelo paintings, fractured religions and a new Scientific method. What will Competence Spread and IoT Data Science “revolution” lead to?!From an abstract point of view, Memory involves more organization in the brain and hence a reduction in entropy. Printed page can hold a lot more “memories” and hence the Printing Press revolution gave us an external way to reduce entropy of “the human system”. Competence is also an exercise in entropy reduction; data get analyzed and organized; insights are drawn. IoT Data Science is very adept at handling tons of Big Data and extracting insights to increase competence; thus, IoT Data Science gives us an external way to reduce entropy.What does such reduction in entropy mean in practical terms? Recognizing that entropy reduction happens for Human+IoT as a *system*, the immediate opportunity will be in empowering the human element with competence augmentation. What I see emerging quickly is, instead of a “personal” assistant, a Work Assistant which is an individualized “machine learner” enhancing our *work* competence which no doubt, will lead each of us to “do more at higher quality”. Beyond that, there is no telling what amazing things “competence-empowered human comprehension” will create . . .I am no Industrial IoT futurist; in the Year 1440, Gutenberg could not have foreseen Michelangelo paintings, fractured religions or a new Scientific method! Similarly, standing here in 2017, it is not apparent what new disruptions IoT revolution will spawn that drop entropy precipitously. I for one am excited about the possibilities and surprises in store in the next few decades.PG Madhavan, Ph.D. - “LEADER . . . of a life in pursuit of excellence . . . in IoT Data Science” http://www.linkedin.com/in/pgmadSee More

]]>