Numerical Linear Algebra
where the algebra stops being the arithmetic
Generator

rank-diagonal

One function in the rrqr library, called 12 times across 8 essays. Below: what it draws at its defaults, what it draws at every value an essay asks for, the 64 claims it put to the test while drawing them, and where it stands against the rule this site is named for.

At its defaults it draws the diagonal of a pivoted r against the singular values, on a 40×40 kahan matrix at c = 0.5. Two curves down the index. The diagonal of R decays at one constant rate to 0.00366; the singular values follow it and then fall away to 9.07·10⁻¹⁰ at the last one. Column pivoting made no interchange at all, because every trailing column norm is equal at every step — the rule had nothing to choose between.

rank-diagonal is one function in lib/figures/rrqr.js — the cheap rank — a greedy pivot rule, and the matrix it has nothing to choose on. Everything below came out of it during this build, at arguments taken from the essays rather than invented for this page. A figure here is the figure a reader meets in an essay, and if the generator changes, this page changes with it.

At its defaults

Drawn even though every essay passes arguments — which on this site is every essay, at 100% of placements since the standard pass. A default nothing exercises is a trap for the next essay to call this with none, and this is the page where a default that has drifted from the figures around it becomes visible.

The diagonal of a pivoted R against the singular values, on a 40×40 Kahan matrix at c = 0.5Two curves down the index. The diagonal of R decays at one constant rate to 0.00366; the singular values follow it and then fall away to 9.07·10⁻¹⁰ at the last one. Column pivoting made no interchange at all, because every trailing column norm is equal at every step — the rule had nothing to choose between.1611162126313610⁻¹⁰10⁻⁸10⁻⁶10⁻⁴10⁻²110²indexmagnitude|r_kk|σ_kone factorisation, two verdicts‖AP − QR‖/‖A‖10⁻¹⁶|r_nn|0.0037σ_min9.1·10⁻¹⁰column interchanges0|r_nn| is never below σ_minso the cheap verdict errs one way only

Two curves down the index. The diagonal of R decays at one constant rate to 0.00366; the singular values follow it and then fall away to 9.07·10⁻¹⁰ at the last one. Column pivoting made no interchange at all, because every trailing column norm is equal at every step — the rule had nothing to choose between.

planted: 12

The arguments are the ones A reflection cannot stop being one passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

The diagonal of a pivoted R against the singular values, on a matrix of planted rank 12Two curves down the index, lying on top of each other. Both fall by more than five orders of magnitude between index 12 and index 13, so both report rank 12 at any threshold placed in the gap.1611162126313610⁻¹³10⁻¹¹10⁻⁹10⁻⁷10⁻⁵10⁻³10⁻¹10¹indexmagnitude|r_kk|σ_kone factorisation, two verdicts‖AP − QR‖/‖A‖10⁻¹⁵|r_nn|1.1·10⁻¹²σ_min10·10⁻¹³column interchanges33|r_nn| is never below σ_minso the cheap verdict errs one way only

Two curves down the index, lying on top of each other. Both fall by more than five orders of magnitude between index 12 and index 13, so both report rank 12 at any threshold placed in the gap.

planted: 0

The arguments are the ones An estimate that can be fooled passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

The diagonal of a pivoted R against the singular values, on a 40×40 Kahan matrix at c = 0.5Two curves down the index. The diagonal of R decays at one constant rate to 0.00366; the singular values follow it and then fall away to 9.07·10⁻¹⁰ at the last one. Column pivoting made no interchange at all, because every trailing column norm is equal at every step — the rule had nothing to choose between.1611162126313610⁻¹⁰10⁻⁸10⁻⁶10⁻⁴10⁻²110²indexmagnitude|r_kk|σ_kone factorisation, two verdicts‖AP − QR‖/‖A‖10⁻¹⁶|r_nn|0.0037σ_min9.1·10⁻¹⁰column interchanges0|r_nn| is never below σ_minso the cheap verdict errs one way only

Two curves down the index. The diagonal of R decays at one constant rate to 0.00366; the singular values follow it and then fall away to 9.07·10⁻¹⁰ at the last one. Column pivoting made no interchange at all, because every trailing column norm is equal at every step — the rule had nothing to choose between.

planted: 24

The arguments are the ones The best approximation there is passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

The diagonal of a pivoted R against the singular values, on a matrix of planted rank 24Two curves down the index, lying on top of each other. Both fall by more than five orders of magnitude between index 24 and index 25, so both report rank 24 at any threshold placed in the gap.1611162126313610⁻¹³10⁻¹¹10⁻⁹10⁻⁷10⁻⁵10⁻³10⁻¹10¹indexmagnitude|r_kk|σ_kone factorisation, two verdicts‖AP − QR‖/‖A‖10⁻¹⁵|r_nn|1.3·10⁻¹²σ_min10·10⁻¹³column interchanges35|r_nn| is never below σ_minso the cheap verdict errs one way only

Two curves down the index, lying on top of each other. Both fall by more than five orders of magnitude between index 24 and index 25, so both report rank 24 at any threshold placed in the gap.

planted: 0, n: 50

The arguments are the ones The cheap rank and what it cannot see passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

The diagonal of a pivoted R against the singular values, on a 50×50 Kahan matrix at c = 0.5Two curves down the index. The diagonal of R decays at one constant rate to 8.69·10⁻⁴; the singular values follow it and then fall away to 3.73·10⁻¹² at the last one. Column pivoting made no interchange at all, because every trailing column norm is equal at every step — the rule had nothing to choose between.171319253137434910⁻¹²10⁻¹⁰10⁻⁸10⁻⁶10⁻⁴10⁻²110²indexmagnitude|r_kk|σ_kone factorisation, two verdicts‖AP − QR‖/‖A‖1.1·10⁻¹⁶|r_nn|8.7·10⁻⁴σ_min3.7·10⁻¹²column interchanges0|r_nn| is never below σ_minso the cheap verdict errs one way only

Two curves down the index. The diagonal of R decays at one constant rate to 8.69·10⁻⁴; the singular values follow it and then fall away to 3.73·10⁻¹² at the last one. Column pivoting made no interchange at all, because every trailing column norm is equal at every step — the rule had nothing to choose between.

planted: 12, n: 50

The arguments are the ones The cheap rank and what it cannot see passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

The diagonal of a pivoted R against the singular values, on a matrix of planted rank 12Two curves down the index, lying on top of each other. Both fall by more than five orders of magnitude between index 12 and index 13, so both report rank 12 at any threshold placed in the gap.171319253137434910⁻¹³10⁻¹¹10⁻⁹10⁻⁷10⁻⁵10⁻³10⁻¹10¹indexmagnitude|r_kk|σ_kone factorisation, two verdicts‖AP − QR‖/‖A‖9.3·10⁻¹⁶|r_nn|10⁻¹²σ_min10·10⁻¹³column interchanges44|r_nn| is never below σ_minso the cheap verdict errs one way only

Two curves down the index, lying on top of each other. Both fall by more than five orders of magnitude between index 12 and index 13, so both report rank 12 at any threshold placed in the gap.

What it checked while drawing

Every figure above asserted its own claims on the way to being drawn, and a claim that failed would have failed the build rather than drawn a wrong picture. Those assertions used to leave no trace at all: a passing one returned true and the only evidence the figure had checked anything was that nothing crashed. The list below is what they actually said, collected by running this generator with an observer installed — not a description of what it is believed to check.

64 distinct claims across 6 sets of arguments, grouped below by shape — because most of them are one sentence with a different number in it, and how many separate times that sentence was put to the test is the informative part.

the diagonal entry at 1 is a magnitude — asserted 50 times

a Kahan parameter strictly inside (0, 1)

a planted rank inside the matrix, or none

a size the decay has room in

a size the SVD can afford

and |r_nn| is at least σ_min, which holds for every triangular factor

and the diagonal reports the planted rank

and the two disagree about σ_min by orders of magnitude

as do the singular values

column pivoting interchanges on a matrix with a planted gap

column pivoting makes no interchange on Kahan's matrix

matmul shapes agree

the factorisation reconstructs

through a gap in the diagonal that is visible

with the diagonal decaying at one constant rate and no gap in it

Against the rule

It draws a decomposition and prints its residual. It calls svd, qrColumnPivoted, and every figure above carries the badge — which residualcheck verifies by looking for it in the emitted SVG rather than by finding the call that builds one. A badge that is constructed and then left out of the body is the failure that check exists for.

Across the library: the rule bites on 90 of 174 generators — 75 print a residual and 15 are exempt with a published reason; 84 factorise nothing. Read from lib/residual-rule.js, which is the same body the gate enforces from, and the gate's last check fails the build if this page and it disagree about any generator.

Where it is called

Changing this generator changes every figure on this list. That is what makes the list worth publishing rather than keeping in a check script.

Orthogonality, measured

A reflection cannot stop being one

Householder QR holds orthogonality at 10⁻¹⁵ whatever the condition number of the matrix, and Gram–Schmidt does not. The reason is not that it is more careful. It is that its Q is built from unit vectors, and rounding a unit vector gives a different reflection rather than a broken one.

 Two errors, and whose fault they are

An estimate that can be fooled

Nobody computes a condition number, because forming an inverse costs more than the solve did. Every library estimates it instead, from four or five products with a factorisation already in hand. The estimate is exactly right on four random matrices out of five — and there is a matrix, three distinct entries wide, on which it returns a twentieth of the truth.

 Orthogonality, measured

Orthogonal is a number

"Q is orthogonal" is a claim about a measurable quantity, ‖QᵀQ − I‖, and on the eight-by-eight Hilbert matrix two standard algorithms return 10⁻¹⁵ and 1 for it. The one that returns 1 still reconstructs the matrix perfectly, which is why nothing warns you.

 Eigenvalues, singular values, rank

Rank is a decision

A floating-point matrix does not have a rank. It has a spectrum of singular values, and somewhere in that spectrum is a place where the values stop being signal and start being noise. Deciding where is a judgement, and the evidence for it is a gap.

 Eigenvalues, singular values, rank

The best approximation there is

The error of the best rank-k approximation is not bounded by the next singular value. It is equal to it. That is an unusually sharp theorem, and it makes the theorem itself usable as an independent check on the computation.

 Eigenvalues, singular values, rank

The cheap rank and what it cannot see

Almost nobody computes singular values to decide a rank. The standard substitute is QR with column pivoting, read off the diagonal of R — and there is a triangular matrix on which the greedy rule makes no interchange at all, has no better column available at any step, and reports a matrix eight orders of magnitude further from singular than it is.

 Least squares, and the road not to take

When the matrix is wrong too

Every least-squares problem on this site has assumed A is exact and b is not, and moved b onto the column space of A. Where both were measured, the smallest correction that makes the system consistent moves the matrix as well — and on the problems where that answer is more accurate, it has the larger residual, by construction rather than by luck.

 Regularisation, and the answer that is chosen

Where the answer stops being in the data

The Picard condition finds the index where a noisy right-hand side stops carrying signal, from the data alone, with no knowledge of the answer. It lands at 32 where the truncation that actually minimises the error is 28 — and at 45 where the best is 38. It overshoots at every noise level from 1% to 0.001%, and it overshoots for a reason.

The whole library · All essays · What must fail