residual-vs-error
At its defaults it draws backward and forward error against the condition number. A log–log plot over twelve decades of condition number. The backward error is a flat line at ten to the minus sixteen; the forward error rises in proportion to the condition number.
residual-vs-error is one function in lib/figures/error.js —
error — the backward one, the forward one, and the number between them. Everything below came out of it during this build, at
arguments taken from the essays rather than invented for this page. A figure here is the
figure a reader meets in an essay, and if the generator changes, this page changes with it.
At its defaults
Drawn even though every essay passes arguments — which on this site is every essay, at 100% of placements since the standard pass. A default nothing exercises is a trap for the next essay to call this with none, and this is the page where a default that has drifted from the figures around it becomes visible.
A log–log plot over twelve decades of condition number. The backward error is a flat line at ten to the minus sixteen; the forward error rises in proportion to the condition number.
size2: 8
The arguments are the ones A reflection cannot stop being one passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.
A log–log plot over twelve decades of condition number. The backward error is a flat line at ten to the minus sixteen; the forward error rises in proportion to the condition number.
What it checked while drawing
Every figure above asserted its own claims on the way to being drawn, and a claim that failed
would have failed the build rather than drawn a wrong picture. Those assertions used to leave
no trace at all: a passing one returned true and the only evidence the figure had
checked anything was that nothing crashed. The list below is what they actually said, collected
by running this generator with an observer installed — not a description of
what it is believed to check.
13 distinct claims across 2 sets of arguments, grouped below by shape — because most of them are one sentence with a different number in it, and how many separate times that sentence was put to the test is the informative part.
κ bounds the forward error at κ = 10 — asserted 3 times
LU is for square matrices
matmul shapes agree
the backward error never leaves the rounding level
the backward-error tolerance sits between the measured noise floor and the smallest real failure
while the forward error climbs by eight orders or more
κ bounds the forward error at κ = 10¹¹
κ bounds the forward error at κ = 10¹³
κ bounds the forward error at κ = 10⁵
κ bounds the forward error at κ = 10⁷
κ bounds the forward error at κ = 10⁹
Against the rule
It calls a factoriser without drawing a factorisation
(solve),
so the rule is written down as not applying, with the reason:
plots two errors against κ — no factor of any matrix is shown
The exemption list is the interesting half of the rule rather than an escape hatch — it is
where a decision about a figure had to be argued in one line. residualcheck
refuses an exemption that is not doing work, and rejected ten of the fifteen written for the
expansion's figures on exactly that ground: a figure whose vertical axis is a residual
satisfies the rule by construction, and touching a factoriser does not by itself require an
entry.
Across the library: the rule bites on 52
of 99 generators —
37 print a residual and
15 are exempt with a published reason;
47 factorise nothing.
Read from lib/residual-rule.js, which is the same body the gate enforces from,
and the gate's last check fails the build if this page and it disagree about any generator.
Where it is called
Changing this generator changes every figure on this list. That is what makes the list worth publishing rather than keeping in a check script.
A reflection cannot stop being one
Householder QR holds orthogonality at 10⁻¹⁵ whatever the condition number of the matrix, and Gram–Schmidt does not. The reason is not that it is more careful. It is that its Q is built from unit vectors, and rounding a unit vector gives a different reflection rather than a broken one.
Two errors, and whose fault they areA small residual is not a small error
Substituting the answer back and finding that it fits is the most natural check there is, and it verifies the wrong thing. A residual of 10⁻¹⁷ is entirely compatible with an answer whose second digit is wrong.
Two errors, and whose fault they areAn answer that is known
Almost every demonstration of numerical error estimates the error by computing the same thing more carefully. The Hilbert matrix does not need that: its inverse is a closed form in integers, so the true answer is available exactly and the error is measured rather than approximated.
Iterating, instead of factorisingAn orthogonalisation nobody calls one
Conjugate gradients are derived as a minimisation and behave as an orthogonalisation, which is why the finite-termination property in every textbook is not a property the method has in floating point.
The arithmetic underneathBuying the accuracy back
Factorise in single precision, then correct the answer using residuals computed in double, and the result is what a full double-precision solve would have given. Compute those residuals in single instead and the identical algorithm, at identical cost, recovers nothing.
The arithmetic underneathCancellation takes the answer, not a digit
Subtracting two nearly equal numbers is exact. That is what makes it dangerous — the subtraction introduces no error at all, it exposes error the operands were already carrying, and the exposure can consume every significant figure at once.
Elimination, and the swapElimination is a sequence of choices
Gaussian elimination is taught as a procedure with no decisions in it. There is one decision at every step — which row to use — and every stability property the algorithm has comes from making it well.
Eigenvalues, singular values, rankRank is a decision
A floating-point matrix does not have a rank. It has a spectrum of singular values, and somewhere in that spectrum is a place where the values stop being signal and start being noise. Deciding where is a judgement, and the evidence for it is a gap.
Eigenvalues, singular values, rankSymmetry is worth more than precision
A symmetric matrix gives up its eigenvalues to full accuracy however ill-conditioned it is. An unsymmetric one can move them by the eighth root of a perturbation, so the rounding involved in merely storing the matrix shifts the spectrum by a hundredth.
Eigenvalues, singular values, rankThe best approximation there is
The error of the best rank-k approximation is not bounded by the next singular value. It is equal to it. That is an unusually sharp theorem, and it makes the theorem itself usable as an independent check on the computation.
Elimination, and the swapThe bound that is never attained
Partial pivoting's stability guarantee permits the entries to double at every step — a factor of 5.5·10¹¹ at n = 40. The measured growth on random matrices of that size is about three. The gap is eleven orders of magnitude, and the guarantee is still worth having.
Two errors, and whose fault they areThe condition number is an amplifier
κ is usually introduced as a definition and then quoted. It is a measurement: perturb the input by a known amount, look at how much the output moves, and the largest ratio you can find is the number.
Two errors, and whose fault they areThe exact answer to a nearby problem
A good algorithm does not give an approximate answer to your problem. It gives the exact answer to a problem very close to yours — and once that is the definition, a wrong result has two possible authors and they can be measured apart.
The arithmetic underneathThe order they are added in
Addition is associative in the algebra and is not associative in the arithmetic. The same million numbers, added in a different order, give answers that differ in the third significant figure — and the fix is not a wider float, it is a different order.
Least squares, and the road not to takeThe projection and the right angle
The least-squares solution is the one whose residual is perpendicular to everything the columns can reach. That is not a mnemonic — it is an equation, Aᵀr = 0, and the computed answer satisfies it to 10⁻¹⁶.
Iterating, instead of factorisingThe rate the condition number predicts
Conjugate gradients converge at a rate governed by the square root of the condition number. That is a bound rather than an estimate, it is provable, and it is loose enough that provisioning iterations from it wastes nine out of ten.
Least squares, and the road not to takeThe road that squares the problem
The normal equations are the first method every course teaches and the method no library uses. Forming AᵀA squares the condition number, and below ε = √u it does not degrade — it produces a matrix that is exactly singular, from data that was perfectly usable.
Least squares, and the road not to takeThe valley with no bottom
A degree-nine fit's coefficients can be moved by a third of their own size before the residual changes in the sixth significant figure. The arithmetic did not lose those digits. The data never contained them.
Regularisation, and the answer that is chosenWhen the answer is a choice
A backward-stable least-squares solve of this problem returns an answer whose relative error is 5.5·10⁸. Nothing went wrong. The singular values decay exponentially with no gap anywhere in them, the data does not determine the answer, and something outside the data has to choose — which is the computation rather than a preliminary to it.