cancellation
At its defaults it draws relative error of two algebraically identical expressions for (1 − cos x)/x², in binary64. A log–log plot of relative error against x. The expression written as it reads loses accuracy as x falls and is eventually wrong in every digit; the rearranged form stays at rounding level.
cancellation is one function in lib/figures/arith.js —
arithmetic — what a float holds, and what it loses holding it. Everything below came out of it during this build, at
arguments taken from the essays rather than invented for this page. A figure here is the
figure a reader meets in an essay, and if the generator changes, this page changes with it.
At its defaults
Drawn even though every essay passes arguments — which on this site is every essay, at 100% of placements since the standard pass. A default nothing exercises is a trap for the next essay to call this with none, and this is the page where a default that has drifted from the figures around it becomes visible.
A log–log plot of relative error against x. The expression written as it reads loses accuracy as x falls and is eventually wrong in every digit; the rearranged form stays at rounding level.
bits: 24
The arguments are the ones A bound that is proved passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.
A log–log plot of relative error against x. The expression written as it reads loses accuracy as x falls and is eventually wrong in every digit; the rearranged form stays at rounding level.
bits: 53
The arguments are the ones Cancellation takes the answer, not a digit passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.
A log–log plot of relative error against x. The expression written as it reads loses accuracy as x falls and is eventually wrong in every digit; the rearranged form stays at rounding level.
bits: 11
The arguments are the ones The numbers below the smallest one passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.
A log–log plot of relative error against x. The expression written as it reads loses accuracy as x falls and is eventually wrong in every digit; the rearranged form stays at rounding level.
bits: 8
The arguments are the ones Where the hardware went passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.
A log–log plot of relative error against x. The expression written as it reads loses accuracy as x falls and is eventually wrong in every digit; the rearranged form stays at rounding level.
What it checked while drawing
Every figure above asserted its own claims on the way to being drawn, and a claim that failed
would have failed the build rather than drawn a wrong picture. Those assertions used to leave
no trace at all: a passing one returned true and the only evidence the figure had
checked anything was that nothing crashed. The list below is what they actually said, collected
by running this generator with an observer installed — not a description of
what it is believed to check.
11 distinct claims across 5 sets of arguments, grouped below by shape — because most of them are one sentence with a different number in it, and how many separate times that sentence was put to the test is the informative part.
and from x = 0.031 down it returns exactly zero — asserted 2 times
the naive form has lost half its digits by x = 0.022 — asserted 2 times
and from x = 1.5·10⁻⁸ down it returns exactly zero
and from x = 3.5·10⁻⁴ down it returns exactly zero
the naive form has lost half its digits by x = 1.1·10⁻⁸
the naive form has lost half its digits by x = 2.4·10⁻⁴
the stable-form tolerance sits between the measured noise floor and the smallest real failure
the two spellings agree where nothing cancels
while the stable form holds to rounding level throughout
Against the rule
The rule does not apply to it. It factorises nothing, so there is no residual it could be withholding. That is worth stating rather than leaving blank: a site that reported the rule as satisfied by every generator would be counting mostly generators the rule never reached.
Across the library: the rule bites on 52
of 99 generators —
37 print a residual and
15 are exempt with a published reason;
47 factorise nothing.
Read from lib/residual-rule.js, which is the same body the gate enforces from,
and the gate's last check fails the build if this page and it disagree about any generator.
Where it is called
Changing this generator changes every figure on this list. That is what makes the list worth publishing rather than keeping in a check script.
A bound that is proved
Every error statement on this site so far is a measurement of one run. Interval arithmetic makes a different kind of claim — the answer lies in this set, for this input, with no probability attached — and its failure mode is that it returns nothing at all. On a Hilbert system it proves a bound 23 times the error it bounds, and one size later it refuses.
The arithmetic underneathA norm that overflows before it is a norm
The vector of sixteen thousands has a Euclidean norm of 4,000, which fp16 represents exactly. Written as the square root of the sum of squares it returns infinity, because squaring doubles the exponent — and the expression costs half the format's range on the one computation every iterative method performs at every step.
The arithmetic underneathBuying the accuracy back
Factorise in single precision, then correct the answer using residuals computed in double, and the result is what a full double-precision solve would have given. Compute those residuals in single instead and the identical algorithm, at identical cost, recovers nothing.
The arithmetic underneathCancellation takes the answer, not a digit
Subtracting two nearly equal numbers is exact. That is what makes it dangerous — the subtraction introduces no error at all, it exposes error the operands were already carrying, and the exposure can consume every significant figure at once.
The arithmetic underneathThe numbers below the smallest one
Below the smallest normal number the spacing stops halving and stays put, all the way to zero. That is what gradual underflow is, and the thing it buys is the sentence every algorithm assumes without being told — x minus y is zero only when x equals y.
The arithmetic underneathThe order they are added in
Addition is associative in the algebra and is not associative in the arithmetic. The same million numbers, added in a different order, give answers that differ in the third significant figure — and the fix is not a wider float, it is a different order.
The arithmetic underneathWhat a float can hold
The representable numbers are not a fine fuzz spread evenly over the line. They are evenly spaced inside each power-of-two interval and twice as far apart in the next one up, and almost everything else in this subject is a consequence of that one fact.
The arithmetic underneathWhere the hardware went
bfloat16 carries eight mantissa bits, which puts its refinement threshold at a condition number of 256. That is not an exotic matrix. It is an ordinary one, and past it the method still improves the answer by a factor of four hundred while getting nowhere near a usable one.