Numerical Linear Algebra — where the algebra stops being the arithmetic Illustrated essays on numerical linear algebra in which no decomposition is drawn without its residual printed: the loss of orthogonality measured rather than described, the error separated into the algorithm's fault and the problem's, and the precision itself on a slider. https://www.numerical-linear-algebra.com/ 2026-08-06T14:00:04.547Z What a float can hold https://www.numerical-linear-algebra.com/essays/what-a-float-can-hold/ 2026-08-06T14:00:04.547Z The representable numbers are not a fine fuzz spread evenly over the line. They are evenly spaced inside each power-of-two interval and twice as far apart in the next one up, and almost everything else in this subject is a consequence of that one fact. The exact answer to a nearby problem https://www.numerical-linear-algebra.com/essays/the-exact-answer-to-a-nearby-problem/ 2026-08-06T14:00:04.547Z A good algorithm does not give an approximate answer to your problem. It gives the exact answer to a problem very close to yours — and once that is the definition, a wrong result has two possible authors and they can be measured apart. Elimination is a sequence of choices https://www.numerical-linear-algebra.com/essays/elimination-is-a-sequence-of-choices/ 2026-08-06T14:00:04.547Z Gaussian elimination is taught as a procedure with no decisions in it. There is one decision at every step — which row to use — and every stability property the algorithm has comes from making it well. Orthogonal is a number https://www.numerical-linear-algebra.com/essays/orthogonal-is-a-number/ 2026-08-06T14:00:04.547Z "Q is orthogonal" is a claim about a measurable quantity, ‖QᵀQ − I‖, and on the eight-by-eight Hilbert matrix two standard algorithms return 10⁻¹⁵ and 1 for it. The one that returns 1 still reconstructs the matrix perfectly, which is why nothing warns you. The projection and the right angle https://www.numerical-linear-algebra.com/essays/the-projection-and-the-right-angle/ 2026-08-06T14:00:04.547Z The least-squares solution is the one whose residual is perpendicular to everything the columns can reach. That is not a mnemonic — it is an equation, Aᵀr = 0, and the computed answer satisfies it to 10⁻¹⁶. Symmetry is worth more than precision https://www.numerical-linear-algebra.com/essays/symmetry-is-worth-more-than-precision/ 2026-08-06T14:00:04.547Z A symmetric matrix gives up its eigenvalues to full accuracy however ill-conditioned it is. An unsymmetric one can move them by the eighth root of a perturbation, so the rounding involved in merely storing the matrix shifts the spectrum by a hundredth. Cancellation takes the answer, not a digit https://www.numerical-linear-algebra.com/essays/cancellation-takes-the-answer/ 2026-08-06T14:00:04.547Z Subtracting two nearly equal numbers is exact. That is what makes it dangerous — the subtraction introduces no error at all, it exposes error the operands were already carrying, and the exposure can consume every significant figure at once. A small residual is not a small error https://www.numerical-linear-algebra.com/essays/a-small-residual-is-not-a-small-error/ 2026-08-06T14:00:04.547Z Substituting the answer back and finding that it fits is the most natural check there is, and it verifies the wrong thing. A residual of 10⁻¹⁷ is entirely compatible with an answer whose second digit is wrong. The swap that is not optional https://www.numerical-linear-algebra.com/essays/the-swap-that-is-not-optional/ 2026-08-06T14:00:04.547Z Run elimination without a row interchange on a matrix that needs one and nothing announces a failure. There is no division by zero, no warning, and an answer of the right shape. It is simply wrong, and how wrong depends on a number you did not look at. Two Gram–Schmidts https://www.numerical-linear-algebra.com/essays/two-gram-schmidts/ 2026-08-06T14:00:04.547Z One argument changes. Classical Gram–Schmidt projects the original column onto each previous direction; modified projects what is left of it. In exact arithmetic the coefficients are identical. In floating point they differ by eight orders of magnitude in the thing that matters. The road that squares the problem https://www.numerical-linear-algebra.com/essays/the-road-that-squares-the-problem/ 2026-08-06T14:00:04.547Z The normal equations are the first method every course teaches and the method no library uses. Forming AᵀA squares the condition number, and below ε = √u it does not degrade — it produces a matrix that is exactly singular, from data that was perfectly usable. Rank is a decision https://www.numerical-linear-algebra.com/essays/rank-is-a-decision/ 2026-08-06T14:00:04.547Z A floating-point matrix does not have a rank. It has a spectrum of singular values, and somewhere in that spectrum is a place where the values stop being signal and start being noise. Deciding where is a judgement, and the evidence for it is a gap. The order they are added in https://www.numerical-linear-algebra.com/essays/the-order-they-are-added-in/ 2026-08-06T14:00:04.547Z Addition is associative in the algebra and is not associative in the arithmetic. The same million numbers, added in a different order, give answers that differ in the third significant figure — and the fix is not a wider float, it is a different order. The condition number is an amplifier https://www.numerical-linear-algebra.com/essays/the-condition-number-is-an-amplifier/ 2026-08-06T14:00:04.547Z κ is usually introduced as a definition and then quoted. It is a measurement: perturb the input by a known amount, look at how much the output moves, and the largest ratio you can find is the number. The bound that is never attained https://www.numerical-linear-algebra.com/essays/the-bound-that-is-never-attained/ 2026-08-06T14:00:04.547Z Partial pivoting's stability guarantee permits the entries to double at every step — a factor of 5.5·10¹¹ at n = 40. The measured growth on random matrices of that size is about three. The gap is eleven orders of magnitude, and the guarantee is still worth having. A reflection cannot stop being one https://www.numerical-linear-algebra.com/essays/a-reflection-cannot-stop-being-one/ 2026-08-06T14:00:04.547Z Householder QR holds orthogonality at 10⁻¹⁵ whatever the condition number of the matrix, and Gram–Schmidt does not. The reason is not that it is more careful. It is that its Q is built from unit vectors, and rounding a unit vector gives a different reflection rather than a broken one. The valley with no bottom https://www.numerical-linear-algebra.com/essays/the-valley-with-no-bottom/ 2026-08-06T14:00:04.547Z A degree-nine fit's coefficients can be moved by a third of their own size before the residual changes in the sixth significant figure. The arithmetic did not lose those digits. The data never contained them. The best approximation there is https://www.numerical-linear-algebra.com/essays/the-best-approximation-there-is/ 2026-08-06T14:00:04.547Z The error of the best rank-k approximation is not bounded by the next singular value. It is equal to it. That is an unusually sharp theorem, and it makes the theorem itself usable as an independent check on the computation. An answer that is known https://www.numerical-linear-algebra.com/essays/an-answer-that-is-known/ 2026-08-06T14:00:04.547Z Almost every demonstration of numerical error estimates the error by computing the same thing more carefully. The Hilbert matrix does not need that: its inverse is a closed form in integers, so the true answer is available exactly and the error is measured rather than approximated.