orth-loss-curve
At its defaults it draws loss of orthogonality against condition number, in binary64. A log–log plot of the norm of Q-transpose-Q minus the identity against condition number. Classical Gram–Schmidt rises steeply, modified Gram–Schmidt rises gently, and Householder is flat.
orth-loss-curve is one function in lib/figures/ortho.js —
orthogonality — ‖qᵀq − i‖ as a measurement rather than an adjective. Everything below came out of it during this build, at
arguments taken from the essays rather than invented for this page. A figure here is the
figure a reader meets in an essay, and if the generator changes, this page changes with it.
At its defaults
Drawn even though every essay passes arguments — which on this site is every essay, at 100% of placements since the standard pass. A default nothing exercises is a trap for the next essay to call this with none, and this is the page where a default that has drifted from the figures around it becomes visible.
A log–log plot of the norm of Q-transpose-Q minus the identity against condition number. Classical Gram–Schmidt rises steeply, modified Gram–Schmidt rises gently, and Householder is flat.
bits: 53
The arguments are the ones A bound that holds with probability passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.
A log–log plot of the norm of Q-transpose-Q minus the identity against condition number. Classical Gram–Schmidt rises steeply, modified Gram–Schmidt rises gently, and Householder is flat.
bits: 24
The arguments are the ones Two Gram–Schmidts passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.
A log–log plot of the norm of Q-transpose-Q minus the identity against condition number. Classical Gram–Schmidt rises steeply, modified Gram–Schmidt rises gently, and Householder is flat.
What it checked while drawing
Every figure above asserted its own claims on the way to being drawn, and a claim that failed
would have failed the build rather than drawn a wrong picture. Those assertions used to leave
no trace at all: a passing one returned true and the only evidence the figure had
checked anything was that nothing crashed. The list below is what they actually said, collected
by running this generator with an observer installed — not a description of
what it is believed to check.
36 distinct claims across 3 sets of arguments, grouped below by shape — because most of them are one sentence with a different number in it, and how many separate times that sentence was put to the test is the informative part.
and both still reconstruct A at κ = 10 — asserted 3 times
classical is never much better than modified at κ = 10 — asserted 3 times
Householder holds orthogonality at κ = 10 — asserted 3 times
and both still reconstruct A at κ = 10¹⁰
and both still reconstruct A at κ = 10¹¹
and both still reconstruct A at κ = 10⁴
and both still reconstruct A at κ = 10⁵
and both still reconstruct A at κ = 10⁶
and both still reconstruct A at κ = 10⁷
and both still reconstruct A at κ = 10⁸
and both still reconstruct A at κ = 10⁹
and Householder ends the range below both of them
classical is never much better than modified at κ = 10¹⁰
classical is never much better than modified at κ = 10¹¹
classical is never much better than modified at κ = 10⁴
classical is never much better than modified at κ = 10⁵
classical is never much better than modified at κ = 10⁶
classical is never much better than modified at κ = 10⁷
classical is never much better than modified at κ = 10⁸
classical is never much better than modified at κ = 10⁹
classical loses orthogonality by orders of magnitude that modified does not
Householder holds orthogonality at κ = 10¹⁰
Householder holds orthogonality at κ = 10¹¹
Householder holds orthogonality at κ = 10⁴
Householder holds orthogonality at κ = 10⁵
Householder holds orthogonality at κ = 10⁶
Householder holds orthogonality at κ = 10⁷
Householder holds orthogonality at κ = 10⁸
Householder holds orthogonality at κ = 10⁹
matmul shapes agree
Against the rule
It calls a factoriser without drawing a factorisation
(qrHouseholder, gramSchmidt),
so the rule is written down as not applying, with the reason:
the whole plot is ‖QᵀQ − I‖ — the residual is the vertical axis
The exemption list is the interesting half of the rule rather than an escape hatch — it is
where a decision about a figure had to be argued in one line. residualcheck
refuses an exemption that is not doing work, and rejected ten of the fifteen written for the
expansion's figures on exactly that ground: a figure whose vertical axis is a residual
satisfies the rule by construction, and touching a factoriser does not by itself require an
entry.
Across the library: the rule bites on 52
of 99 generators —
37 print a residual and
15 are exempt with a published reason;
47 factorise nothing.
Read from lib/residual-rule.js, which is the same body the gate enforces from,
and the gate's last check fails the build if this page and it disagree about any generator.
Where it is called
Changing this generator changes every figure on this list. That is what makes the list worth publishing rather than keeping in a check script.
A bound that holds with probability
Every other guarantee on this site is deterministic. The randomised low-rank approximation offers one that holds with a probability, the seed changes the answer, and the honest figure is a band rather than a line.
Where the flop count stopped predicting the timeA reduction that changes the order
A tall-skinny QR computed as a tree of independent block factorisations touches a 512×12 matrix once instead of twelve times, computes a completely different sequence of roundings from the sweep it replaces, and returns ‖AᵀA − RᵀR‖/‖AᵀA‖ = 1.65·10⁻¹⁵ against the sweep's 9.95·10⁻¹⁵. On the same matrix classical Gram–Schmidt returns 4.6·10⁻¹⁰.
Orthogonality, measuredA reflection cannot stop being one
Householder QR holds orthogonality at 10⁻¹⁵ whatever the condition number of the matrix, and Gram–Schmidt does not. The reason is not that it is more careful. It is that its Q is built from unit vectors, and rounding a unit vector gives a different reflection rather than a broken one.
Iterating, instead of factorisingAn orthogonalisation nobody calls one
Conjugate gradients are derived as a minimisation and behave as an orthogonalisation, which is why the finite-termination property in every textbook is not a property the method has in floating point.
Orthogonality, measuredOrthogonal is a number
"Q is orthogonal" is a claim about a measurable quantity, ‖QᵀQ − I‖, and on the eight-by-eight Hilbert matrix two standard algorithms return 10⁻¹⁵ and 1 for it. The one that returns 1 still reconstructs the matrix perfectly, which is why nothing warns you.
Eigenvalues, singular values, rankRank is a decision
A floating-point matrix does not have a rank. It has a spectrum of singular values, and somewhere in that spectrum is a place where the values stop being signal and start being noise. Deciding where is a judgement, and the evidence for it is a gap.
Eigenvalues, singular values, rankThe algorithm the libraries actually run
Factorise, multiply the factors back in the other order, repeat. That description is complete and correct and produces something nobody would use — on a matrix with eigenvalues +1 and −1 it does not converge at all, and the subdiagonal entry does not move by so much as a rounding error.
Eigenvalues, singular values, rankThe best approximation there is
The error of the best rank-k approximation is not bounded by the next singular value. It is equal to it. That is an unusually sharp theorem, and it makes the theorem itself usable as an independent check on the computation.
Two errors, and whose fault they areThe condition number is an amplifier
κ is usually introduced as a definition and then quoted. It is a measurement: perturb the input by a known amount, look at how much the output moves, and the largest ratio you can find is the number.
Randomised, and the guarantee that changes kindThe dimension does not appear
A random projection preserves the lengths of a set of vectors to within a distortion that depends on how many vectors there are and not on how many coordinates each one has. That is the fact the whole field rests on, and it is genuinely surprising.
Eigenvalues, singular values, rankThe form that makes it affordable
One Householder reduction, done once, turns every subsequent iteration of the eigenvalue algorithm from cubic to quadratic cost. It changes no answer at all, which is why it is easy to describe as an optimisation and wrong to.
Least squares, and the road not to takeThe projection and the right angle
The least-squares solution is the one whose residual is perpendicular to everything the columns can reach. That is not a mnemonic — it is an equation, Aᵀr = 0, and the computed answer satisfies it to 10⁻¹⁶.
Least squares, and the road not to takeThe road that squares the problem
The normal equations are the first method every course teaches and the method no library uses. Forming AᵀA squares the condition number, and below ε = √u it does not degrade — it produces a matrix that is exactly singular, from data that was perfectly usable.
Iterating, instead of factorisingThe spectrum that predicts nothing
For a symmetric matrix the eigenvalues govern how fast an iteration converges. Drop symmetry and they stop governing anything — there is a matrix whose eigenvalues are as evenly spread as eigenvalues can be, on which GMRES makes no progress at all until the last possible step.
Elimination, and the swapThe swap that is not optional
Run elimination without a row interchange on a matrix that needs one and nothing announces a failure. There is no division by zero, no warning, and an answer of the right shape. It is simply wrong, and how wrong depends on a number you did not look at.
Least squares, and the road not to takeThe valley with no bottom
A degree-nine fit's coefficients can be moved by a third of their own size before the residual changes in the sixth significant figure. The arithmetic did not lose those digits. The data never contained them.
Orthogonality, measuredTwo Gram–Schmidts
One argument changes. Classical Gram–Schmidt projects the original column onto each previous direction; modified projects what is left of it. In exact arithmetic the coefficients are identical. In floating point they differ by eight orders of magnitude in the thing that matters.