Every essay
Grouped by field, below. Or come at it from what is taught and refused here — every claim settled by an assertion that would have failed the build; or from the fields themselves, the depth ladders, or the figure library, where every generator publishes what it checked before it drew.
The arithmetic underneath 26Two errors, and whose fault they are 26Elimination, and the swap 26Orthogonality, measured 26Least squares, and the road not to take 26Regularisation, and the answer that is chosen 26Eigenvalues, singular values, rank 32The eigenvalue problem that is not linear 26Reduction, and what a model is for 25Iterating, instead of factorising 39Sparsity, and what elimination costs 26Structure, and the solver that cannot see it 26Neither sparse nor dense 26When the index is a tuple 24The matrix that is a graph 26Exact arithmetic, and what it costs instead 26Randomised, and the guarantee that changes kind 26Methods that were designed apart 25When the problem arrives again 26Where the flop count stopped predicting the time 26The answer that depends on the machine 26The matrix a constraint makes 25
The arithmetic underneath
The representable numbers are spaced, and the spacing doubles at every power of two. Everything else follows from that: why subtracting two close numbers destroys the part you wanted, why the order of a sum changes its value, and why the mantissa is the one parameter on this site worth putting on a slider.
What a float can hold
The representable numbers are not a fine fuzz spread evenly over the line.
Cancellation takes the answer, not a digit
Subtracting two nearly equal numbers is exact.
The order they are added in
Addition is associative in the algebra and is not associative in the arithmetic.
Buying the accuracy back
Factorise in single precision, then correct the answer using residuals computed in double, and the result is what a full double-precision solve would have given.
Where the hardware went
bfloat16 carries eight mantissa bits, which puts its refinement threshold at a condition number of 256.
The other half of a format
fp16 and tf32 have the same eleven significand bits and their largest numbers are 65,504 and 3.4·10³⁸.
A norm that overflows before it is a norm
The vector of sixteen thousands has a Euclidean norm of 4,000, which fp16 represents exactly.
The numbers below the smallest one
Below the smallest normal number the spacing stops halving and stays put, all the way to zero.
Eight bits, and a format that breaks the rules
E4M3 reuses the exponent code IEEE reserves for infinities, so it reaches 448 where the same bits under IEEE's rules would reach 240 — and has no infinity left to signal an overflow with.
The direction the error leans
The size of one rounding error is set by the precision.
A coin flip that fixes the average
Add 0.1 to 256 a thousand times at eight significand bits and the answer is 256.
One exponent for thirty-two numbers
Share the exponent across a block and the cost per value drops from eight bits to 6.25, and the accuracy improves — up to about three octaves of spread inside a block.
A bit buys an octave
The outlier a block survives is exactly two raised to its significand width — 8 at three bits, 32 at five, 128 at seven, 512 at nine.
A bound that is proved
Every error statement in the essays before this one is a measurement of one run.
Proving the answer is in the box
Every other method here computes a number and estimates how wrong it is.
Where the box is cut
A branch-and-bound with an interval operator settles a whole square — two roots proved unique, forty-two regions proved empty, nothing left undecided, in 87 evaluations.
Three walks and one bound
A left-to-right sum, a chain of three thousand rotations and a conjugate gradient residual recurrence share no arithmetic and no vocabulary.
The units that overflow before the answer does
A change of variable that is exact in the algebra requires γ² times a matrix to be a number the format can hold.
Nine steps of pessimism
A proved bound is 8 to 26 times the error it bounds, at every precision from 16 to 40 significand bits.
The well on the far side of the band
Gradual underflow was said to buy a predicate and not an answer, because a quantity that has decayed into the subnormal range is already lost.
The far well in a byte
Half precision's subnormals carried a two-well chain's far well ten octaves deeper than flush to zero, and the essay that measured it predicted that on the eight-bit formats the band would be worth almost nothing and starting high almost everything.
Room for the step, not the value
Starting a recurrence at the top of an eight-bit format's range spends its headroom once.
The formula sets the power, the sum sets the constant
The one-pass variance was found losing three to sixty-nine times more than the digits-lost rule allows, and the explanation offered — operands built from a thousand roundings — was never priced.
The generator that rounded its own product
A one-pass variance leaned the same way on forty-one seeds, and the low bits of a congruential generator were the suspect.