The nine fields
The arithmetic underneath
The representable numbers are spaced, and the spacing doubles at every power of two. Everything else follows from that: why subtracting two close numbers destroys the part you wanted, why the order of a sum changes its value, and why the mantissa is the one parameter on this site worth putting on a slider.
12 essays — What a float can hold, Cancellation takes the answer, not a digit, The order they are added in and 9 more
Two errors, and whose fault they are
A good algorithm returns the exact answer to a nearby problem. So a wrong answer has two possible authors, and they can be measured separately: the backward error says how well the algorithm did, the condition number says how much the problem amplifies it, and only their product is what anyone sees.
4 essays — The exact answer to a nearby problem, A small residual is not a small error, The condition number is an amplifier and 1 more
Elimination, and the swap
Gaussian elimination is the first algorithm anybody learns and the row interchange is the part nobody is given a reason for. Here it is run without one, on a matrix where that fails — quietly, returning an answer of the right shape — and the growth factor that governs the whole story is plotted against a bound it never attains.
3 essays — Elimination is a sequence of choices, The swap that is not optional, The bound that is never attained
Orthogonality, measured
Orthogonal is not an adjective, it is a number: ‖QᵀQ − I‖. Two algorithms that are the same algebra written in a different order return 10⁻¹⁵ and 1 for it on the same matrix — and the one that fails still reconstructs the matrix perfectly, which is why nothing warns you.
3 essays — Orthogonal is a number, Two Gram–Schmidts, A reflection cannot stop being one
Least squares, and the road not to take
The normal equations are taught first and used by nobody, because forming AᵀA squares the condition number and then, below a computable value of ε, breaks outright. Underneath that is a harder fact: a wide range of very different fits explain the data equally well, and no arithmetic can choose between them.
3 essays — The projection and the right angle, The road that squares the problem, The valley with no bottom
Eigenvalues, singular values, rank
A symmetric matrix gives up its eigenvalues without complaint. An unsymmetric one can move them by the eighth root of a perturbation, so rounding error alone shifts them by a hundredth. And rank is not a property a floating-point matrix has — it is a decision about a gap, and the gap is worth printing beside it. The algorithm that libraries actually run lives here too, and it needs a shift before it is an algorithm at all.
10 essays — Symmetry is worth more than precision, Rank is a decision, The best approximation there is and 7 more
Iterating, instead of factorising
At scale nobody factorises, and the reason is not accuracy — it is that the factors of a sparse matrix are not sparse. What replaces elimination is a sequence of approximations, and the question changes shape: not what the residual of the factorisation is, but how fast the error falls and whether the thing you can measure tracks the thing you cannot. On the model problem every rate is known in closed form before anything runs.
14 essays — The rate the condition number predicts, An orthogonalisation nobody calls one, Changing the condition number on purpose and 11 more
Sparsity, and what elimination costs
Eliminating a variable couples everything it touched to everything else it touched, and every one of those couplings is an entry that was zero in the matrix and is not zero in its factor. On the same matrix one elimination order gives a factor of a thousand entries and another gives ten thousand — the two factorisations equally accurate, one of them fitting in memory. Nothing numerical chooses between them.
6 essays — The factor is not sparse, The order decides the memory, Two ends of the same arrow and 3 more
Randomised, and the guarantee that changes kind
Every other claim on this site is deterministic: the residual is below a bound or it is not. A randomised low-rank approximation has an error bound that holds with a probability, and the seed changes the answer — so the figures draw a band rather than a line. What sketching buys is the same quality for fewer passes over the data, which is a statement about cost. On a matrix with no structure to find it buys nothing, and neither does anything else.
3 essays — A bound that holds with probability, The dimension does not appear, Randomisation does not create structure
Every essay, with its figures · The threads that cross the fields · The depth ladders