Numerical Linear Algebra
where the algebra stops being the arithmetic
Concept

Silent failure — where it appears

A computation that returns an answer of the right shape and the wrong value, with every quantity the caller can measure reporting success. It is the failure mode this whole collection is organised around, because every quantity a caller can measure reports success and the answer is still wrong.

Named by 6 essays across 3 fields — each of them below, with the objects they name alongside it.

[ ε 1 ; 1 1 ] x = [ 1 ; 2 ], exact answer (1.000000, 1.000000)with partial pivoting1101U after elimination1.0000001.000000computed xbackward error 0forward error 0without10⁻¹⁷10-1·10¹⁷U after elimination0.0000001.000000computed xbackward error 0.25forward error 0.71no error is raisedgrowth 10¹⁷

The swap that is not optional

Run elimination without a row interchange on a matrix that needs one and nothing announces a failure. There is no division by zero, no warning, and an answer of the right shape. It is simply wrong, and how wrong depends on a number you did not look at.

elimination · Pivoting forward error, relative to a solution of exactly (1, 1)no pivoting · as given1 0 interchangesno pivoting · rows scaled1 0 interchangespartial · as given0 1 interchangepartial · rows scaled1 0 interchangesscaled partial · as given0 1 interchangescaled partial · rows scaled0 1 interchangecomplete · as given0 1 interchangecomplete · rows scaled0 1 interchangethe same problem twicepartial, as given10⁻¹⁸partial, rows scaled1its relative residual10⁻¹⁷complete, rows scaled10⁻¹⁸the two systems have the same solutionand one pivot rule cannot see it

The pivot that reads the units

Partial pivoting compares the entries of a column and takes the largest. Those entries carry units, so the comparison depends on them — and there is a row scaling, on the standard two-by-two that pivoting exists to fix, which makes partial pivoting perform the identical catastrophic elimination it was introduced to prevent, with no interchange at all.

elimination · Pivoting the estimator maximises this quantity over the columns it visitscolumn 1 ‹visited›12column 2 ‹the answer›114column 311.4column 411.4column 511.4column 611.4column 711.4column 811.4column 911.4column 1011.4column 1111.4column 1211.4estimate 12.0a walk that stopped earlythe estimate returned12the true 1-norm114columns visited1products with the matrix5the walk's own stopping test firedand every column it could see was smaller

An estimate that can be fooled

Nobody computes a condition number, because forming an inverse costs more than the solve did. Every library estimates it instead, from four or five products with a factorisation already in hand. The estimate is exactly right on four random matrices out of five — and there is a matrix, three distinct entries wide, on which it returns a twentieth of the truth.

error · Condition-estimation -15-13-11-910⁻¹³10⁻¹¹10⁻⁹10⁻⁷log₁₀ of the defect, relative to the entry it sits inhow far the answer movedthe machine, on its owna tolerance with two sidesmachine band3.2·10⁻¹²smallest caught10⁻¹²its nearest run4.6·10⁻¹²defects hidden3window, factor1.4below the band nothing is visibleand above it everything is

What a regression test can ask for

The machine's own variation on one solve is 3.2·10⁻¹², and the smallest defect whose answers clear it is one part in 10¹². The tolerance exists, it is bracketed on both sides by a factor of 1.42, and it is neither zero nor the 10⁻⁸ that usually gets typed.

machine · Regression tolerance each bar is a percentage — of the sample, or of the true condition numberexactly right86.0%inside 10%91.0%inside a factor of 291.0%worst in the sample, ×10035.7%the constructed matrix, ×1007.7%usually exactexact share0.86worst of the sample0.36the constructed matrix0.077a routine that is right most of the timeand never wrong in the safe direction

The tail a sample never reaches

Hager's estimator is exactly right on four random matrices in five, and that share is stable — between 80.5 and 87.5 per cent across nine sizes. The worst underestimate is not stable at all: it falls every time more matrices are drawn, from 0.746 at sixty to 0.377 at four hundred, and the matrix built to defeat the estimator sits five times below anything four hundred draws found.

error · Condition-estimation LAPACK's walk, exact83.0% · 5.2 productsLAPACK's walk, worst × 10037.7% · 5.2 productsblock of one, exact84.0% · 4.4 productsblock of one, worst × 10037.7% · 4.4 productsblock of two, exact96.5% · 8.5 productsblock of two, worst × 10059.6% · 8.5 productsblock of four, exact100.0% · 16.8 productsblock of four, worst × 100100.0% · 16.8 productseach bar a percentage — of the sample, or of the truthtwo random vectors close most of the tail

Two columns see what one walk cannot

The condition estimator every library ships walks from the all-ones vector, and a matrix whose largest column cancels against that vector hides from it: at n = 24 it reports five per cent of the truth. The block estimator behind MATLAB's condest walks with two vectors, the second random. On the same matrix at three sizes it is exact on every one of twenty seeds. On four hundred random 8 × 8 matrices it is exact on 96.5 per cent where the single walk is exact on 83.0, and its worst case, 0.596, is reached in the first fifty draws and not lowered by the next 1,550. The single walk's worst was still falling at 1,600. Four vectors are exact on all 400.

error · Condition-estimation

Named alongside it

The objects these essays reach for when they reach for this one.

Backward errorCondition-estimationCondition numberCounterexampleLower boundMatrix normWorst-case analysisGaussian eliminationGrowth factorPartial pivotingSeeded generatorBackward stability

All concepts