Concept

Orthonormal basis — where it appears

A set of vectors of unit length that are mutually perpendicular and span a space. Coordinates in it are inner products, it has condition number one, and computing one for the columns of a matrix is what Gram–Schmidt, Householder and Givens factorisations do.

Named by 2 essays across 2 fields — each of them below, with the objects they name alongside it.

The basis nobody chose on purpose

A method that eliminates a constraint has to pick a basis for its null space, and every basis is correct. Their condition numbers are eight orders apart, the reduced problem inherits the square, and the choice is usually made by a one-line rule nobody thought of as a numerical decision.

orthogonality · Null-space basis

The degree that is safe to overshoot

The rules that choose a Tikhonov parameter miss by factors of millions on one draw in twenty. Transplanted to the degree of a polynomial fit, in a basis orthonormal on the data, the same rules never cost more than 2.7 times the best degree's error in three hundred draws. The reason is the shape of the valley they search: six degrees too few costs from 44 to 16,000 times the best error, forty degrees too many costs about twice it. The one rule with a tail, the discrepancy principle, has its threshold half a standard deviation above the residual it is waiting for.

leastsquares · Fitting

Named alongside it

The objects these essays reach for when they reach for this one.

Basis Bias variance Column pivoting Condition number Constrained minimisation Cross-validation Discrepancy principle Generalised cross-validation Least-squares Leverage Null-space Null-space basis

All concepts