Replies to This Discussion

Permalink Reply by Dr. Dimitrios Geromichalos on March 27, 2016 at 11:22pm

Thanks for the article, it raises some interesting questions.

I would guess, prediction is just a special case of extrapolation concerning a special variable (time) and direction (future). Admittedly, it's the most commonly used case (especially in finance) but there other types of extrapolation, too (e.g. in engineering).

Another very important issue you mentioned are the confidence bands. Especially in finance, predictions are never accurate but lie always in confidence bands. Here, the "bad" cases are normally more interesting than the "good" ones since they determine the possible survival (or bankruptcy) of financial institutions. The whole field of (quantitative) risk management basically deals with the "edges" of these confidence bands.

I worked myself several years in the field of risk management and think lot of concepts developed there can be useful in data science and vice versa. I've summarized some of them in an article some time ago:

http://www.analyticbridge.com/profiles/blogs/what-can-quantitative-...

• ▶ Reply

Permalink Reply by Desmond Brennan on March 28, 2016 at 8:41am

There's an additional problem as well , one that is common to most social sciences models

The observed ...is aware that they are being observed

So if you publish any result they may either start striving towards it ... or "game the system" in order not to reach it

• ▶ Reply

Permalink Reply by Douglas A Dame on March 28, 2016 at 10:21am

Predictive modeling includes both interpolation and extrapolation.

I have always used the term "extrapolation" as a technical term meaning "Beware!" Not sure where I picked that habit up. But it's intended to telegraph the fact that by operating outside the range of known data, we are adding a huge, untestable assumption into the mix ... namely that the data relationships found in the known ranges for the inputs are also valid outside those ranges.

For time series predictions extending into the future, the caveats about extrapolation are implied.

HTH

• ▶ Reply

Permalink Reply by Kwaku Damoah on March 29, 2016 at 5:05pm

In practice extrapolation/interpolation has been used when looking at single variables with fewer data but prediction or predictive modeling usually involves a larger number of variables and data points.

• ▶ Reply

Permalink Reply by Valentin Malakhov on March 29, 2016 at 10:00pm

Kwaku is right. Extra-interpolation is the method of creating more "apples" from existing "apples". Both are guessing the values either within an interval or beyond. Extrapolation " right-ward" can be seen as prediction. But I see prediction as a more complicated operation of creating "apples" from apples, pears, time, money, space and whatever the person can imagine. Prediction has a time flavour (what happens in future) and does not imply any certain method (in difference to the other two).

• ▶ Reply

Permalink Reply by nauman sheikh on April 6, 2016 at 8:06am

Prediction in the AI space is very different. In the AI space Forecasting (through extrapolation) is different from Prediction Since prediction would require the same event to have occurred in the past to learn from and then based on new data points predict the likelihood of that event. For example:

Forecasting Question: What will be the temperature tomorrow. Forecasting Answer: 65 degree - based on extrapolation of historical data.

Prediction Qs: What is the likelihood that it will be over 60 degree tomorrow? Prediction Answer: (once a model has been trained with historic data) there is a 75% probability that it will be over 60 degrees tomorrow.

• ▶ Reply

Permalink Reply by Bogdan Gherman on May 5, 2016 at 1:23pm

I agree with you Laetitia, I belive the diference between the two terms is just semantics, in effect they are the same.
In my view "prediction" implies a machine learning algorithm is used whereas "extrapolation" implies for me the more direct statistical methods...
BTW: Very good question!
All the best,
Bogdan

• ▶ Reply

Permalink Reply by Theodore Omtzigt on September 7, 2016 at 3:05am

If you include simulation models in the predictive camp, then extrapolation and predictive modeling are very different. With simulation modeling, you create predictions from first principles. Most of the supercomputing cycles in the world are occupied with predictive modeling of biochemical, fusion, and fission reactions, none of them extrapolate from known data.

• ▶ Reply

Permalink Reply by Jacob Zahavi on September 11, 2016 at 5:27am

Reply by Jacob Zahavi

There’s quite a difference between interpolation/extrapolation and prediction. The former belongs to the realm of explanatory models, the latter to the realm of predictive analytics.

What is the difference?

Explanatory models, often involving linear regression, are concerned with explaining a given phenomenon and finding causal relationships between an output (dependent) variable, and a host, often very few, input (independent) variables. The objective is to find a good regression model that fits the data very well which meets the underlying assumption of linear regression. The emphasis here is on hypothesis testing, p-values, confidence intervals,…Once a good model is found, one can use it for estimating the value of the output variable for given values of the input variables. It is OK to estimate an output value based on interpolation, but one must use extreme caution in estimating output values based on extrapolation because the regression model is an explanatory model, not a predictive one.

Predictive models, on the other hand, are concerned with predicting the output values of new observations. While regression models may also be harnessed for prediction, they are quite different than explanatory models. The objective is not to find a regression model with perfect fit, but to find a model with high prediction power and no over fitting, even if it requires relaxing the significance level of introducing/excluding input variables from the model. Prediction models use observed and measurable variables, usually many of them, which require the use of advanced feature selection techniques to render a model with high prediction power that generalizes well for new observations. The prediction accuracy is usually assessed by means of a holdout (validation) sample rather than by means of checking the validity of the model assumptions and testing of hypotheses.

Indeed, prediction models and explanatory models are not the same, being less strict about model assumptions, and focusing instead on prediction accuracy and over-fitting. More on the differences between explanatory models and prediction models can be found in the paper by Shmueli, G., and O. Koppius, "Predictive Analytics in Information Systems Research", MIS Quarterly, vol. 35, issue 3, pp. 553-572, 2011.

 

 

• ▶ Reply