Errors-in-variables — where it appears
Named by 4 essays across one field — each of them below, with the objects they name alongside it.
Also named here as total least squares — the same set of essays touches all of them, so they are one junction rather than several.
When the matrix is wrong too
Every least-squares problem here has assumed A is exact and b is not, and moved b onto the column space of A. Where both were measured, the smallest correction that makes the system consistent moves the matrix as well — and on the problems where that answer is more accurate, it has the larger residual, by construction rather than by luck.
The two numbers a caller has
Choosing between the two least-squares methods is a statement about where the noise is, and the two quantities a caller can compute are both blind to it. The residual separates the answers by 0.14 per cent where their accuracies differ by 14, and κ(A) falls from 3.54 to 2.46 across a sweep in which the error rises by a factor of sixty-two.
A unit is a statement about the noise
Total least squares minimises one Frobenius norm over every column of [A b], so the unit a column is written in is a claim about how noisy it is. On a 60-row fit with two measured regressors, rewriting one column in units from 10⁻⁴ to 10⁴ times the recorded ones leaves ordinary least squares exactly where it was and moves total least squares by half again, continuously, between two estimators with their own names: the reverse regression of that column on the others, and the fit that treats it as exact. Its correction is split among the columns as the squares of the coefficients, which the units set and the noise never enters. Dividing each column by its noise level makes every unit give one answer, the best of five estimators on all three placements of the noise tried, and two replicate readings a column are enough to get most of the way there when the recorded units were badly wrong.
The weights the fit chooses for itself
Total least squares needs each column's noise level, and the tempting shortcut is to read it off the fit: weight, fit, take each column's correction as its noise, reweight, repeat. The prediction was that this converges, to weights that depend on where it starts. It does not converge at all. Each step multiplies every weight by its own column's share of the smallest singular vector, the one self-consistent weighting is unstable — a departure from it doubles every step — and every run leaves for the edge in two to seven steps, landing on the reverse regression or on ordinary least squares by which side it started. The self-consistent weights are nearly the same whatever the noise is: they describe the fit's geometry, and only replicate readings describe the noise.
Named alongside it
The objects these essays reach for when they reach for this one.
Least squaresTotal least squaresSingular value decompositionCondition numberExact ground truthForward errorNoise levelResidualScalingWeighted least squaresEckart–YoungFixed-point iteration