Just curious about the following: model fitting for polynomial regression, and least squares outlier issues (over-fitting). In which contexts does polynomial regression make sense? And how to make it robust? Is it better to first transform the data before doing any kind of regression? And how to choose the best transformation?
I think it is useful for curve fitting, but perhaps cubic splines are easier to work with.
I use multivariate polynomials to solve the classification problem in a multivariate feature space. It is like a regression, with a few extra layers of complexity.
You can find a paper here: https://github.com/adaviding/Morpe
Particle physicists (phenomenology/experimentalist) definitely uses it a lot to describe particle interactions.
This is a strange question. There are lots of places where polynomial models describe important physical phenomena. Like gravity. Here's a good use case, known as "Avenge Kill". Your sensors make a bunch of positional measurements on an incoming mortar shell, and you want to fire back. You fit a 2nd order polynomial model to the points, and fire on the (other) location where that 2nd order polynomial intersects the Earth's surface. Why second order polynomial? Because the physics is second order polynomial. So any other model would be silly.
In short, you use a polynomial model where you have good reason to believe that the underlying "physics" are polynomial. As for data transformation, I assume you mean linearization. To me, this usually doesn't make sense. There are a couple of exceptions:
As a vaguely theoretical note, Chebyshev Polynomials are an exceedingly good fitting model. That has little to do with least squares, but the fact that Chebyshev Polynomials work so well suggests that polynomials are powerful models.