precision-ladder
At its defaults it draws forward error of a 6×6 hilbert solve at eight precisions. A bar for each significand width from 12 to 53 bits showing the relative error in the computed solution, with the condition number times the unit roundoff marked as a prediction.
precision-ladder is one function in lib/figures/arith.js —
arithmetic — what a float holds, and what it loses holding it. Everything below came out of it during this build, at
arguments taken from the essays rather than invented for this page. A figure here is the
figure a reader meets in an essay, and if the generator changes, this page changes with it.
At its defaults
Drawn even though every essay passes arguments — which on this site is every essay, at 100% of placements since the standard pass. A default nothing exercises is a trap for the next essay to call this with none, and this is the page where a default that has drifted from the figures around it becomes visible.
A bar for each significand width from 12 to 53 bits showing the relative error in the computed solution, with the condition number times the unit roundoff marked as a prediction.
n: 6
The arguments are the ones A small residual is not a small error passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.
A bar for each significand width from 12 to 53 bits showing the relative error in the computed solution, with the condition number times the unit roundoff marked as a prediction.
What it checked while drawing
Every figure above asserted its own claims on the way to being drawn, and a claim that failed
would have failed the build rather than drawn a wrong picture. Those assertions used to leave
no trace at all: a passing one returned true and the only evidence the figure had
checked anything was that nothing crashed. The list below is what they actually said, collected
by running this generator with an observer installed — not a description of
what it is believed to check.
11 distinct claims across 2 sets of arguments, grouped below by shape — because most of them are one sentence with a different number in it, and how many separate times that sentence was put to the test is the informative part.
at 12 bits the error is within the κ·u prediction — asserted 8 times
and the backward error stays small at every precision, which is the point of the figure
LU is for square matrices
the ladder actually descends
Against the rule
It calls a factoriser without drawing a factorisation
(solve),
so the rule is written down as not applying, with the reason:
solves a system at eight precisions; the figure is the error, not the factors
The exemption list is the interesting half of the rule rather than an escape hatch — it is
where a decision about a figure had to be argued in one line. residualcheck
refuses an exemption that is not doing work, and rejected ten of the fifteen written for the
expansion's figures on exactly that ground: a figure whose vertical axis is a residual
satisfies the rule by construction, and touching a factoriser does not by itself require an
entry.
Across the library: the rule bites on 52
of 99 generators —
37 print a residual and
15 are exempt with a published reason;
47 factorise nothing.
Read from lib/residual-rule.js, which is the same body the gate enforces from,
and the gate's last check fails the build if this page and it disagree about any generator.
Where it is called
Changing this generator changes every figure on this list. That is what makes the list worth publishing rather than keeping in a check script.
A small residual is not a small error
Substituting the answer back and finding that it fits is the most natural check there is, and it verifies the wrong thing. A residual of 10⁻¹⁷ is entirely compatible with an answer whose second digit is wrong.
Two errors, and whose fault they areAn answer that is known
Almost every demonstration of numerical error estimates the error by computing the same thing more carefully. The Hilbert matrix does not need that: its inverse is a closed form in integers, so the true answer is available exactly and the error is measured rather than approximated.
The arithmetic underneathBuying the accuracy back
Factorise in single precision, then correct the answer using residuals computed in double, and the result is what a full double-precision solve would have given. Compute those residuals in single instead and the identical algorithm, at identical cost, recovers nothing.
Two errors, and whose fault they areThe condition number is an amplifier
κ is usually introduced as a definition and then quoted. It is a measurement: perturb the input by a known amount, look at how much the output moves, and the largest ratio you can find is the number.
The arithmetic underneathThe order they are added in
Addition is associative in the algebra and is not associative in the arithmetic. The same million numbers, added in a different order, give answers that differ in the third significant figure — and the fix is not a wider float, it is a different order.
The arithmetic underneathWhat a float can hold
The representable numbers are not a fine fuzz spread evenly over the line. They are evenly spaced inside each power-of-two interval and twice as far apart in the next one up, and almost everything else in this subject is a consequence of that one fact.
The arithmetic underneathWhere the hardware went
bfloat16 carries eight mantissa bits, which puts its refinement threshold at a condition number of 256. That is not an exotic matrix. It is an ordinary one, and past it the method still improves the answer by a factor of four hundred while getting nowhere near a usable one.