Numerical Linear Algebra
where the algebra stops being the arithmetic
Concept

Weighted least-squares — where it appears

A least-squares fit in which each residual is multiplied by a weight before the squares are summed, usually the inverse of that measurement's noise. Weighting by the noise makes the fit independent of the units each column was recorded in, which plain least squares already is and total least squares is not.

Named by 2 essays across 2 fields — each of them below, with the objects they name alongside it.

10⁻¹101020304050z, the effective noise over thicknesstrials mirrored, %5 points, equal20 points, equal80 points, equal● five precise, 1/σ²■ five weighted ×100per cent mirrored at z = 0.45, 1,000 trialstwenty points1.8eighty points1.3five points9.5five precise, weighted 1/σ²7.9equal noise, five weighted ×1006.8both weighted schemes have five effective pointstwenty and eighty points share one curve

Five precise points are five points

Weighting each sighting by its reliability is the standard form of an attitude or registration fit, and it changes how often the nearest orthogonal matrix comes back as a mirror. Measured, the rate is a function of two numbers: the weighted noise over thickness, and the effective count (Σw)²/Σw². Five points with a tenth of the noise, weighted by 1/σ², carry the information of 515 equal points and mirror like five — 7.9 per cent at a noise ratio where twenty points mirror 1.8 and five mirror 9.5. The √m the earlier measurement left unchecked is right, and it counts what carries the thin direction.

orthogonality · Polar decomposition the same in every columnleast squares0.32total, weighted0.11total, c → 00.16total, c → ∞0.1710⁻⁴10⁻²110²10⁴00.050.10.150.20.250.30.35factor c the first column's units are changed byrelative error, medianreverse regressionrecorded unitsleast squarescolumn 1 exactweighted by the noisedots: plain total least squaresthe units choose the answer

A unit is a statement about the noise

Total least squares minimises one Frobenius norm over every column of [A b], so the unit a column is written in is a claim about how noisy it is. On a 60-row fit with two measured regressors, rewriting one column in units from 10⁻⁴ to 10⁴ times the recorded ones leaves ordinary least squares exactly where it was and moves total least squares by half again, continuously, between two estimators with their own names: the reverse regression of that column on the others, and the fit that treats it as exact. Its correction is split among the columns as the squares of the coefficients, which the units set and the noise never enters. Dividing each column by its noise level makes every unit give one answer, the best of five estimators on all three placements of the noise tried, and two replicate readings a column are enough to get most of the way there when the recorded units were badly wrong.

leastsquares · Total least-squares

Named alongside it

The objects these essays reach for when they reach for this one.

Singular value decompositionDeterminantErrors-in-variablesExact ground truthLeast-squaresNoise levelOrthogonalityPolar decompositionReflectionRotationScalingTotal least-squares

All concepts