Numerical Linear Algebra
where the algebra stops being the arithmetic
Concept

Errors-in-variables — where it appears

The model in which the predictors are measured with error as well as the responses, so both the matrix and the right-hand side of a fit carry noise. It is the model total least squares is the maximum-likelihood estimator for, and under it an ordinary least-squares estimate is biased towards zero by an amount no quantity of data removes.

Named by 2 essays across one field — each of them below, with the objects they name alongside it.

Also named here as total least-squares — the same set of essays touches all of them, so they are one junction rather than several.

00.250.50.75110⁻¹110¹share of the noise placed in the matrixleast-squares error ÷ total least-squares errorequally accuratetotal leastsquares aheadordinary leastsquares aheadthe model, not the methodadvantage, all noise in b0.28advantage, all noise in A2.5seeds at each share40the same total noise at every pointand only where it sits changes

When the matrix is wrong too

Every least-squares problem on this site has assumed A is exact and b is not, and moved b onto the column space of A. Where both were measured, the smallest correction that makes the system consistent moves the matrix as well — and on the problems where that answer is more accurate, it has the larger residual, by construction rather than by luck.

leastsquares · Total least-squares the upper pair is distance from the truth; the lower pair is ‖Ax − b‖least squares · error0.08523total least squares · error0.037least squares · ‖Ax − b‖3.965total least squares · ‖Ax − b‖4.078two orderingserror ratio (ls ÷ tls)2.3residual ratio (tls ÷ ls)1seeds40no vector makes the residual smallernot even the one the problem was built from

The two numbers a caller has

Choosing between the two least-squares methods is a statement about where the noise is, and the two quantities a caller can compute are both blind to it. The residual separates the answers by 0.14 per cent where their accuracies differ by 14, and κ(A) falls from 3.54 to 2.46 across a sweep in which the error rises by a factor of sixty-two.

leastsquares · Total least-squares

Named alongside it

The objects these essays reach for when they reach for this one.

Condition numberForward errorLeast-squaresResidualSingular value decompositionTotal least-squaresEckart–YoungExact ground truthNormal equationsOrthogonal projection

All concepts