About

This is a growing collection of illustrated essays about numerical linear algebra. Each takes a single claim and draws it until the argument is visible — and every figure showing a factorisation prints the number that says whether the factorisation is what it claims to be.

Why the numerical half

The geometric explanation of linear algebra — a matrix as a transformation, the determinant as an area, the eigenvectors as the directions that do not turn — is superbly covered elsewhere, most of all by 3Blue1Brown's Essence of Linear Algebra. Building a slightly different version of that would be entering a contest that is already settled, and settled fairly.

What is not covered is what happens when those exact identities are executed by a machine whose numbers have gaps in them. QᵀQ is the identity in the algebra. In the arithmetic it is measurably not, and how far from it depends on which of two algebraically identical algorithms you used. That gap is large, it is practical, and almost nobody plots it.

Stated in one sentence, which is the test any such site has to pass: this is the site that shows you where the algebra stops being the arithmetic, and whose fault it is.

Two errors, not one

The organising idea here is an identity that is usually stated once in a lecture and then never used:

forward error ⪅ condition number × backward error

A good algorithm returns the exact answer to a nearby problem. The backward error says how nearby — that is the algorithm's contribution, and for the methods used here it sits at the level of rounding no matter how hard the problem is. The condition number says how much the problem amplifies a small perturbation, and that has nothing to do with the algorithm at all. What the user sees is the product.

So a wrong answer on this site always comes with an attribution. Solving a thirteen-by-thirteen Hilbert system gives components that are wrong by 200%, and the backward error of that solve is 10⁻¹⁷. Nothing is wrong with the elimination. The problem was never going to survive.

The rule

No decomposition is drawn without its residual printed. ‖A − QR‖/‖A‖ beside every QR, ‖QᵀQ − I‖ beside every claim of orthogonality, ‖PA − LU‖/‖A‖ beside every elimination. The numbers are on the figures, in the same place every time.

It is enforced rather than intended: a build in which a generator produces a factorisation and does not print a residual fails, and the exceptions — figures that factorise something on the way to plotting something else — are listed one by one with the reason. That list is the part that matters, because a rule with a silent escape hatch is a convention, and conventions last about fourteen figures.

An answer that is known, not estimated

Most demonstrations of floating-point error estimate the error by computing the same thing more carefully and calling that the truth. It is circular, and it is why so much writing on the subject ends with “as you can see, the error is small”.

The Hilbert matrix does not need that. Its entries are unit fractions, its inverse has a closed form whose every entry is an integer, and exact rational arithmetic in BigInt rounds nowhere. So for a whole class of problems here the true answer is the integers, and the error in a floating-point solve is measured against it rather than approximated. Where two exact routes exist — elimination in rationals, and multiplication by the closed-form inverse — they are required to agree exactly, not to a tolerance.

Precision is the parameter

The decompositions are written against an arithmetic context rather than against + and *, so the identical code runs at any significand width from sixteen bits to fifty-three. That is what lets the mantissa be a slider: drag it and watch the loss of orthogonality slide with it.

Every frame is generated at build time by the same function that drew the static picture, so every assertion the figure makes has been checked at every position the reader can reach — including the positions where the algorithm fails, which are the interesting end. The simulation is checked against the hardware: rounding to twenty-four bits agrees with the CPU's own single precision on every one of twenty thousand test values, which is a second route rather than a plausibility argument.

Two routes to every number, and assertions that can fail

A determinant from the product of the pivots and from cofactor expansion. A singular value from one-sided Jacobi and from the eigenvalues of AᵀA. Eigenvalues that must sum to the trace and multiply to the determinant. Neither route can confirm itself, and the pairs share no arithmetic.

And every check is fed something it must refuse. The orthogonality test is shown a shear, the reconstruction test is shown a factorisation of the wrong matrix, the growth measurement is shown Wilkinson's matrix, which attains a bound that random matrices miss by eleven orders of magnitude. A build in which any of them accepts its counterexample does not complete. An assertion that cannot fail is not a test.

What the site does not do

It explains how these methods behave. It is not a numerical library, and no figure here should be read as a recommendation about production code — the implementations are written out precisely because the subject is their failure modes, which is the opposite of what you want from software you depend on. Use LAPACK.

Everything is generated. There are no stock images, no traced diagrams, and no figures fitted to make a point. Where a claim turned out to be false when measured, the measurement is what got published: the tolerance on one figure had to be loosened because the noise floor was the assembly's rather than the algorithm's, and that is written into the code where it happened rather than quietly fixed.