I myself do epidemiologic research, which rarely calls for developing machine learning models. Instead, I spend my time developing logistic regression models that I have to be able to interpret for the broader scientific (and sometimes non-scientific) community. I have to be able to explain in some easy, risk-communication way not only the…

If you were to invest in real estate, what would be the most important factor you’d take into account? Would it be the age of the building, its location, or maybe how many owners it previously had?

While all of the above will certainly be important to you (though to varying extents), there’s one universal factor that will either be a deal-breaker or make you want to go all in.

You’ve guessed it — the price.

We all have a rough overview of…

Summary:  Bias in modeling has long been a public concern that is now amplified and focused on the disparate treatment models may cause for African Americans.  Defining and correcting the bias presents difficult issues for data scientists that need to be carefully thought through before reaching conclusions.

 …

After being confined for 2 months and hearing the word pandemic almost daily, seeing graphs, opinions, time lines and values that rise and fall, I became curious. Firstly, what is a pandemic? Is it the same as an epidemic? A plague? Was the plague an epidemic? What do epidemiologists study? Yes, they study a discipline called epidemiology.

What is epidemiology?

Epidemiology studies the distribution, frequency, relationships, predictions, and…

How to maintain Model Effectiveness after deployed

When we ready to deploy a good predictive model which given a good accuracy score on training and testing dataset, there is one more problem to solve.  How long will this model effectively solve the problem with the same high accuracy and what is the strategy to maintain the model accuracy. We also need to…

This is a longer form followup to my post describing the open source pandemic package on PyPI (with Python code also available on Github). You can use the code to simulate millions of people moving about in two dimensions, crossing paths and, unfortunately, getting sick.

This video illustrates the dynamic with a toy sized town of fifty people. Watch how transmission takes place. You might even discern commuting and households.…

Machine Learning (ML) development is an iterative process in which the accuracy of predictions made by the models is continuously improved by repeating the training and evaluation phases. In each of these iterations, certain parameters are tweaked continuously by developers. Any parameter manually selected based on learning from previous experiments qualify to be called a model hyper-parameter. These parameters represent intuitive decisions whose value cannot be estimated from data or from…

Summary:  Data Scientists from Booking.com share many lessons learned in the process of constantly improving their sophisticated ML models.  Not the least of which is that improving your models doesn’t always lead to improving business outcomes.

The adoption of AI/ML in business is at an inflection point. …

Summary:  Finally there are tools that let us transcend ‘correlation is not causation’ and identify true causal factors and their relative strengths in our models.  This is what prescriptive analytics was meant to be.

Just when I thought we’d figured it all out, something comes along to make…

Summary:  Whether you’re a data scientist building an implementation case to present to executives or a non-data scientist leader trying to figure this out there’s a need for a much broader framework of strategic thinking around how to capture the value of AI/ML.

There are many articles written from a tools…

Research fields usually follow the practice of categorizing continuous predictor variables, and they are the same who mostly use ANOVA. They often do it through median splits, the high value above the median and the low values below the median. However; this it seems is not that good an idea, and enlisted are some of the reasons to it:

• Median tends to vary from sample to sample. This makes the categories in various samples have various…

I have been writing about the Crosswave Differential Algorithm for a number of years. I described in previous blogs how the algorithm emerged almost by accident while I was attempting to write an application intended to support quality control. In this blog I will be discussing the event model that powers the algorithm. Events are the details and circumstances…

Introduction

There’s a lot of buzzword around the term “Sentiment Analysis” and the various ways of doing it. Great! So you report with reasonable accuracies what the sentiment about a particular brand or product is.

Opinion Mining and Sentiment Analysis

After publishing this report, your client comes back to you and…

Relation, Relationship and Association

While most players in the IT sector adopted Graph or Document databases and Hadoop based solutions, Hadoop is an enabler of HBase column store, it went almost unnoticed that several new DBMS, AtomicDB previous database engine of …

Currently, many of us are overwhelmed with mighty power of Deep Learning. We start to forget about humble graphical models. CRF is not so trendy as LSTM, but it is robust, reliable and worth noting.

In this post, you will find a short summary about CRF (aka Conditional Random Fields) – what is this thing, what is it for and some interesting facts. Enjoy!…

Regression is the first technique you’ll learn in most analytics books. It is a very useful and simple form of supervised learning used to predict a quantitative response.

Originally published on Ideatory…

"Look, they are one people, and they have all one language; and this is only the beginning of what they will do; nothing that they propose to do will now be impossible for them.  Come, let us go down, and confuse their language there, so that they will not understand one another's speech."  (Genesis 11-6,7)  On the distance between expression, meaning, and action resulting from growth of populations and…

This article was first posted in 2014 but the message bears repeating.  There is a lot being written about tools simple enough for the citizen data scientist to operate.  The unstated constraint is that if you don't have significant experience in data science then these will always be "good enough" models.  The problem is that 'good enough' models under achieve both revenue and profit.  Very small increases in model fitness can translate into much larger increases in campaign ROI.  Business…