Numerical Linear Algebra
where the algebra stops being the arithmetic
Generator

projected-residual

One function in the resgap library, called 6 times across 2 essays. Below: what it draws at its defaults, what it draws at every value an essay asks for, the 9 claims it put to the test while drawing them, and where it stands against the rule this site is named for.

At its defaults it draws what gmres reports and what its answer's residual is, on a matrix with one eigenvalue at 1e-14. GMRES also prints a residual it never computes from its answer — but it does not carry one. The number falls out of the Givens rotations at each step as the residual of the projected least-squares problem, re-derived from the whole basis rather than updated from the last value. Over 40 steps the two never separate by more than a factor of 1.80, and they do that while ‖VᵀV − I‖ for the Arnoldi basis is 1.41 — which is to say the basis has stopped being orthogonal and the residual computed from it is still honest.

projected-residual is one function in lib/figures/resgap.js — residual gap — the number a method reports against the residual of its answer. Everything below came out of it during this build, at arguments taken from the essays rather than invented for this page. A figure here is the figure a reader meets in an essay, and if the generator changes, this page changes with it.

At its defaults

Drawn even though every essay passes arguments — which on this site is every essay, at 100% of placements since the standard pass. A default nothing exercises is a trap for the next essay to call this with none, and this is the page where a default that has drifted from the figures around it becomes visible.

What GMRES reports and what its answer's residual is, on a matrix with one eigenvalue at 1e-14GMRES also prints a residual it never computes from its answer — but it does not carry one. The number falls out of the Givens rotations at each step as the residual of the projected least-squares problem, re-derived from the whole basis rather than updated from the last value. Over 40 steps the two never separate by more than a factor of 1.80, and they do that while ‖VᵀV − I‖ for the Arnoldi basis is 1.41 — which is to say the basis has stopped being orthogonal and the residual computed from it is still honest.051015202530354010⁻³10⁻¹10¹GMRES steprelative residualthe two curves are the same curvea number re-derived, not carriedworst reported/actual factor1.8at step40‖VᵀV − I‖ of the basis1.4reported at the last step0.041actual at the last step0.074the same family of methodsand only one of them lies

GMRES also prints a residual it never computes from its answer — but it does not carry one. The number falls out of the Givens rotations at each step as the residual of the projected least-squares problem, re-derived from the whole basis rather than updated from the last value. Over 40 steps the two never separate by more than a factor of 1.80, and they do that while ‖VᵀV − I‖ for the Arnoldi basis is 1.41 — which is to say the basis has stopped being orthogonal and the residual computed from it is still honest.

which: "tiny"

The arguments are the ones The number that is re-derived passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

What GMRES reports and what its answer's residual is, on a matrix with one eigenvalue at 1e-14GMRES also prints a residual it never computes from its answer — but it does not carry one. The number falls out of the Givens rotations at each step as the residual of the projected least-squares problem, re-derived from the whole basis rather than updated from the last value. Over 40 steps the two never separate by more than a factor of 1.80, and they do that while ‖VᵀV − I‖ for the Arnoldi basis is 1.41 — which is to say the basis has stopped being orthogonal and the residual computed from it is still honest.051015202530354010⁻³10⁻¹10¹GMRES steprelative residualthe two curves are the same curvea number re-derived, not carriedworst reported/actual factor1.8at step40‖VᵀV − I‖ of the basis1.4reported at the last step0.041actual at the last step0.074the same family of methodsand only one of them lies

GMRES also prints a residual it never computes from its answer — but it does not carry one. The number falls out of the Givens rotations at each step as the residual of the projected least-squares problem, re-derived from the whole basis rather than updated from the last value. Over 40 steps the two never separate by more than a factor of 1.80, and they do that while ‖VᵀV − I‖ for the Arnoldi basis is 1.41 — which is to say the basis has stopped being orthogonal and the residual computed from it is still honest.

which: "bidiagonal"

The arguments are the ones The number that is re-derived passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

What GMRES reports and what its answer's residual is, on a matrix with non-normal, superdiagonal 30GMRES also prints a residual it never computes from its answer — but it does not carry one. The number falls out of the Givens rotations at each step as the residual of the projected least-squares problem, re-derived from the whole basis rather than updated from the last value. Over 40 steps the two never separate by more than a factor of 2.86, and they do that while ‖VᵀV − I‖ for the Arnoldi basis is 4.48·10⁻¹⁰ — which is to say the basis has stopped being orthogonal and the residual computed from it is still honest.051015202530354010⁻⁶10⁻⁴10⁻²1GMRES steprelative residualthe two curves are the same curvea number re-derived, not carriedworst reported/actual factor2.9at step38‖VᵀV − I‖ of the basis4.5·10⁻¹⁰reported at the last step7.9·10⁻⁵actual at the last step1.6·10⁻⁴the same family of methodsand only one of them lies

GMRES also prints a residual it never computes from its answer — but it does not carry one. The number falls out of the Givens rotations at each step as the residual of the projected least-squares problem, re-derived from the whole basis rather than updated from the last value. Over 40 steps the two never separate by more than a factor of 2.86, and they do that while ‖VᵀV − I‖ for the Arnoldi basis is 4.48·10⁻¹⁰ — which is to say the basis has stopped being orthogonal and the residual computed from it is still honest.

which: "jordan"

The arguments are the ones The number that is re-derived passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

What GMRES reports and what its answer's residual is, on a matrix with a single Jordan blockGMRES also prints a residual it never computes from its answer — but it does not carry one. The number falls out of the Givens rotations at each step as the residual of the projected least-squares problem, re-derived from the whole basis rather than updated from the last value. Over 40 steps the two never separate by more than a factor of 2.00, and they do that while ‖VᵀV − I‖ for the Arnoldi basis is 1.41 — which is to say the basis has stopped being orthogonal and the residual computed from it is still honest.051015202530354010⁻¹⁶10⁻¹³10⁻¹⁰10⁻⁷10⁻⁴10⁻¹GMRES steprelative residualthe two curves are the same curvea number re-derived, not carriedworst reported/actual factor2at step40‖VᵀV − I‖ of the basis1.4reported at the last step1.5·10⁻¹⁵actual at the last step3·10⁻¹⁵the same family of methodsand only one of them lies

GMRES also prints a residual it never computes from its answer — but it does not carry one. The number falls out of the Givens rotations at each step as the residual of the projected least-squares problem, re-derived from the whole basis rather than updated from the last value. Over 40 steps the two never separate by more than a factor of 2.00, and they do that while ‖VᵀV − I‖ for the Arnoldi basis is 1.41 — which is to say the basis has stopped being orthogonal and the residual computed from it is still honest.

What it checked while drawing

Every figure above asserted its own claims on the way to being drawn, and a claim that failed would have failed the build rather than drawn a wrong picture. Those assertions used to leave no trace at all: a passing one returned true and the only evidence the figure had checked anything was that nothing crashed. The list below is what they actually said, collected by running this generator with an observer installed — not a description of what it is believed to check.

9 distinct claims across 4 sets of arguments, grouped below by shape — because most of them are one sentence with a different number in it, and how many separate times that sentence was put to the test is the informative part.

a matrix the comparison is defined on

a matrix the GMRES comparison is defined on

a run long enough to compare

a size the whole run is affordable at

an Arnoldi recurrence started from a vector that is not zero

an eigenvalue small enough to make the iterates large

GMRES never reports a residual a fifth of its answer's

matmul shapes agree

while the basis those residuals are projected onto has lost orthogonality

Against the rule

The rule does not apply to it. It factorises nothing, so there is no residual it could be withholding. That is worth stating rather than leaving blank: a site that reported the rule as satisfied by every generator would be counting mostly generators the rule never reached.

Across the library: the rule bites on 113 of 238 generators — 98 print a residual and 15 are exempt with a published reason; 125 factorise nothing. Read from lib/residual-rule.js, which is the same body the gate enforces from, and the gate's last check fails the build if this page and it disagree about any generator.

Where it is called

Changing this generator changes every figure on this list. That is what makes the list worth publishing rather than keeping in a check script.

Iterating, instead of factorising

The number that is re-derived

GMRES prints a residual it never computes from its answer either. On the matrix that sends a conjugate gradient recurrence 7.3·10¹⁰ wrong, and on two others chosen to be worse, its number is never more than a factor of 2.86 out — while the basis it is computed from has lost orthogonality entirely. The disease is not iterative methods, and it is not floating point.

 Iterating, instead of factorising

The residual the method reports

Conjugate gradients prints a relative residual of 6.9·10⁻²¹. The unit roundoff is 1.1·10⁻¹⁶, so that is not a small residual and not a large one — it is not a residual. The vector the method is holding at that step has ‖b − Ax‖/‖b‖ = 5.1·10⁻¹⁰, and nothing in the run says so.

The whole library · All essays · What must fail