Numerical Linear Algebra
where the algebra stops being the arithmetic
Concept

Basis choice — where it appears

The decision of which set of functions or vectors a problem is written in, when several span the same space. The span fixes the answer; the basis fixes the conditioning of the matrix it is computed from, and so how much rounding and noise reach the result.

Named by 2 essays across one field — each of them below, with the objects they name alongside it.

Also named here as the vandermonde matrix — the same set of essays touches all of them, so they are one junction rather than several.

10⁻⁴10⁻³10⁻²10⁻¹110¹10²10³10⁻¹⁶10⁻¹³10⁻¹⁰10⁻⁷10⁻⁴10⁻¹relative change in the coefficients, along the worst directionrelative increase in the residualcoefficients doubled39% change, fit unmoved in the sixth digit308×: the third digit movesκ(A) = 3.6·10⁶. Exact arithmetic would pick one point on this floor. It would not raise it.24 points, degree 9, monomial basisthe data leaves them free

The valley with no bottom

A degree-nine fit's coefficients can be moved by a third of their own size before the residual changes in the sixth significant figure. The arithmetic did not lose those digits. The data never contained them.

leastsquares · Fitting 08162432404810⁻¹10²10⁵10⁸10¹¹10¹⁴10¹⁷10²⁰degreeκ of the design matrix1/u: past this a double holds nothingmonomialsChebyshevArnoldi on the pointsκ at degree 48monomials1.5·10¹⁷Chebyshev1.6·10⁷Arnoldi on the points1200 two intervals with a gap between themthree bases for one space of polynomials

A basis built from the points

A polynomial fit computed in monomials and in an orthogonal basis gives the same curve on exact data, and the valley essay drew the two lying on top of each other. Add 0.1% noise and they separate — by 1.7·10⁻⁵ at degree 40 and 0.004 at degree 48 — because the fitted curve moves with the basis by its condition number times the rounding times the residual. Chebyshev polynomials keep that small only on points spread like their weight; on a sample with a hole in it they reach κ = 1.55·10⁷. A basis orthogonalised against the sample points themselves stays at 1 on every set.

leastsquares · Fitting

Named alongside it

The objects these essays reach for when they reach for this one.

Condition numberResidualThe Vandermonde matrixArnoldi iterationBackward stabilityChebyshev basisIll-posednessLeast-squaresOrthogonal projectionRegularisationReorthogonalisationSingular values

All concepts