Who still uses polynomial regression?

Just curious about the following: model fitting for polynomial regression, and least squares outlier issues (over-fitting). In which contexts does polynomial regression make sense? And how to make it robust? Is it better to first transform the data before doing any kind of regression? And how to choose the best transformation?

0 members like this

▶ Reply to This

Replies to This Discussion

Permalink Reply by David Ing on June 25, 2015 at 7:33am

I think it is useful for curve fitting, but perhaps cubic splines are easier to work with.

I use multivariate polynomials to solve the classification problem in a multivariate feature space. It is like a regression, with a few extra layers of complexity.

You can find a paper here: https://github.com/adaviding/Morpe

▶ Reply

Permalink Reply by Thanh Hien Mai on June 29, 2015 at 6:56pm

Particle physicists (phenomenology/experimentalist) definitely uses it a lot to describe particle interactions.

▶ Reply

Permalink Reply by Jamie Lawson on October 8, 2015 at 1:44pm

This is a strange question. There are lots of places where polynomial models describe important physical phenomena. Like gravity. Here's a good use case, known as "Avenge Kill". Your sensors make a bunch of positional measurements on an incoming mortar shell, and you want to fire back. You fit a 2nd order polynomial model to the points, and fire on the (other) location where that 2nd order polynomial intersects the Earth's surface. Why second order polynomial? Because the physics is second order polynomial. So any other model would be silly.

In short, you use a polynomial model where you have good reason to believe that the underlying "physics" are polynomial. As for data transformation, I assume you mean linearization. To me, this usually doesn't make sense. There are a couple of exceptions:

If you are only concerned with a local estimator, then transforming to a linear model makes sense; every smooth function is approximately linear at any point, within some neighborhood.
Continuous exponential growth. The exponentials a*e^(rt) is a two parameter family like the 1-D linear functions. And with exponential behavior, linearization reduces the overweighting of data points at the right (assuming positive growth). One way to think about this is that if you get more meaning by showing the data on a log chart than on a standard chart, consider transforming by taking the log of the ordinates.
Where the squared error sum minimization cannot be solved by available methods. The squared error sum minimization problem has an analytic solution for a linear model. For some other models, the problem doesn't have an analytic solution but can be solved by Newton iteration or something similar. There are some pathological problems where standard optimization methods fail to converge. There, you're screwed, but if you can transform the problem you might not be totally screwed. This case is basically trying to make the best of failure.

As a vaguely theoretical note, Chebyshev Polynomials are an exceedingly good fitting model. That has little to do with least squares, but the fact that Chebyshev Polynomials work so well suggests that polynomials are powerful models.