Numerical Linear Algebra
where the algebra stops being the arithmetic
Generator

truncated-cg

One function in the ttrain library, called 8 times across 3 essays. Below: what it draws at its defaults, what it draws at every value an essay asks for, the 4 claims it put to the test while drawing them, and where it stands against the rule this site is named for.

At its defaults it draws conjugate gradients on the three-dimensional model problem with every iterate cut to rank 3. The falling curves are relative residuals: the lower one is the same method with no budget, which reaches 1.61·10⁻¹⁴ in 40 steps, and the upper one is the budgeted run, which stops at 3.85·10⁻⁴. The two step curves near the top are ranks, on their own scale: the un-truncated step asks for 5 at every step from the third onwards and the budget allows 3. The truncation is therefore not an occasional tidy-up — it is happening at every step, and the distance between the two residual curves is what it costs. The solution of this problem is itself a train of rank five, so a budget of five or more removes nothing and the two curves coincide.

truncated-cg is one function in lib/figures/ttrain.js — the train — one rank per cut of the index list, and the iterate that must be cut back. Everything below came out of it during this build, at arguments taken from the essays rather than invented for this page. A figure here is the figure a reader meets in an essay, and if the generator changes, this page changes with it.

At its defaults

Drawn even though every essay passes arguments — which on this site is every essay, at 100% of placements since the standard pass. A default nothing exercises is a trap for the next essay to call this with none, and this is the page where a default that has drifted from the figures around it becomes visible.

Conjugate gradients on the three-dimensional model problem with every iterate cut to rank 3The falling curves are relative residuals: the lower one is the same method with no budget, which reaches 1.61·10⁻¹⁴ in 40 steps, and the upper one is the budgeted run, which stops at 3.85·10⁻⁴. The two step curves near the top are ranks, on their own scale: the un-truncated step asks for 5 at every step from the third onwards and the budget allows 3. The truncation is therefore not an occasional tidy-up — it is happening at every step, and the distance between the two residual curves is what it costs. The solution of this problem is itself a train of rank five, so a budget of five or more removes nothing and the two curves coincide.081624324010⁻¹⁴10⁻¹¹10⁻⁸10⁻⁵10⁻²conjugate gradient steprelative residualabove, on their own scale: rank asked 5, rank kept 3solid: the budgeted residual · light: no budgetbudget 3rank asked for5rank kept3residual, budgeted3.8·10⁻⁴residual, unbudgeted1.6·10⁻¹⁴numbers stored150the step asks for moreat every step

The falling curves are relative residuals: the lower one is the same method with no budget, which reaches 1.61·10⁻¹⁴ in 40 steps, and the upper one is the budgeted run, which stops at 3.85·10⁻⁴. The two step curves near the top are ranks, on their own scale: the un-truncated step asks for 5 at every step from the third onwards and the budget allows 3. The truncation is therefore not an occasional tidy-up — it is happening at every step, and the distance between the two residual curves is what it costs. The solution of this problem is itself a train of rank five, so a budget of five or more removes nothing and the two curves coincide.

maxRank: 3

The arguments are the ones An iterate that must be made smaller passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

Conjugate gradients on the three-dimensional model problem with every iterate cut to rank 3The falling curves are relative residuals: the lower one is the same method with no budget, which reaches 1.61·10⁻¹⁴ in 40 steps, and the upper one is the budgeted run, which stops at 3.85·10⁻⁴. The two step curves near the top are ranks, on their own scale: the un-truncated step asks for 5 at every step from the third onwards and the budget allows 3. The truncation is therefore not an occasional tidy-up — it is happening at every step, and the distance between the two residual curves is what it costs. The solution of this problem is itself a train of rank five, so a budget of five or more removes nothing and the two curves coincide.081624324010⁻¹⁴10⁻¹¹10⁻⁸10⁻⁵10⁻²conjugate gradient steprelative residualabove, on their own scale: rank asked 5, rank kept 3solid: the budgeted residual · light: no budgetbudget 3rank asked for5rank kept3residual, budgeted3.8·10⁻⁴residual, unbudgeted1.6·10⁻¹⁴numbers stored150the step asks for moreat every step

The falling curves are relative residuals: the lower one is the same method with no budget, which reaches 1.61·10⁻¹⁴ in 40 steps, and the upper one is the budgeted run, which stops at 3.85·10⁻⁴. The two step curves near the top are ranks, on their own scale: the un-truncated step asks for 5 at every step from the third onwards and the budget allows 3. The truncation is therefore not an occasional tidy-up — it is happening at every step, and the distance between the two residual curves is what it costs. The solution of this problem is itself a train of rank five, so a budget of five or more removes nothing and the two curves coincide.

maxRank: 1

The arguments are the ones An iterate that must be made smaller passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

Conjugate gradients on the three-dimensional model problem with every iterate cut to rank 1The falling curves are relative residuals: the lower one is the same method with no budget, which reaches 1.61·10⁻¹⁴ in 40 steps, and the upper one is the budgeted run, which stops at 0.119. The two step curves near the top are ranks, on their own scale: the un-truncated step asks for 5 at every step from the third onwards and the budget allows 1. The truncation is therefore not an occasional tidy-up — it is happening at every step, and the distance between the two residual curves is what it costs. The solution of this problem is itself a train of rank five, so a budget of five or more removes nothing and the two curves coincide.081624324010⁻¹⁴10⁻¹¹10⁻⁸10⁻⁵10⁻²conjugate gradient steprelative residualabove, on their own scale: rank asked 5, rank kept 1solid: the budgeted residual · light: no budgetbudget 1rank asked for5rank kept1residual, budgeted0.12residual, unbudgeted1.6·10⁻¹⁴numbers stored30the step asks for moreat every step

The falling curves are relative residuals: the lower one is the same method with no budget, which reaches 1.61·10⁻¹⁴ in 40 steps, and the upper one is the budgeted run, which stops at 0.119. The two step curves near the top are ranks, on their own scale: the un-truncated step asks for 5 at every step from the third onwards and the budget allows 1. The truncation is therefore not an occasional tidy-up — it is happening at every step, and the distance between the two residual curves is what it costs. The solution of this problem is itself a train of rank five, so a budget of five or more removes nothing and the two curves coincide.

maxRank: 5

The arguments are the ones An iterate that must be made smaller passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

Conjugate gradients on the three-dimensional model problem with every iterate cut to rank 5The falling curves are relative residuals: the lower one is the same method with no budget, which reaches 1.61·10⁻¹⁴ in 40 steps, and the upper one is the budgeted run, which stops at 1.61·10⁻¹⁴. The two step curves near the top are ranks, on their own scale: the un-truncated step asks for 5 at every step from the third onwards and the budget allows 5. The truncation is therefore not an occasional tidy-up — it is happening at every step, and the distance between the two residual curves is what it costs. The solution of this problem is itself a train of rank five, so a budget of five or more removes nothing and the two curves coincide.081624324010⁻¹⁴10⁻¹¹10⁻⁸10⁻⁵10⁻²conjugate gradient steprelative residualabove, on their own scale: rank asked 5, rank kept 5solid: the budgeted residual · light: no budgetbudget 5rank asked for5rank kept5residual, budgeted1.6·10⁻¹⁴residual, unbudgeted1.6·10⁻¹⁴numbers stored350the step asks for moreat every step

The falling curves are relative residuals: the lower one is the same method with no budget, which reaches 1.61·10⁻¹⁴ in 40 steps, and the upper one is the budgeted run, which stops at 1.61·10⁻¹⁴. The two step curves near the top are ranks, on their own scale: the un-truncated step asks for 5 at every step from the third onwards and the budget allows 5. The truncation is therefore not an occasional tidy-up — it is happening at every step, and the distance between the two residual curves is what it costs. The solution of this problem is itself a train of rank five, so a budget of five or more removes nothing and the two curves coincide.

maxRank: 2

The arguments are the ones An iterate that must be made smaller passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

Conjugate gradients on the three-dimensional model problem with every iterate cut to rank 2The falling curves are relative residuals: the lower one is the same method with no budget, which reaches 1.61·10⁻¹⁴ in 40 steps, and the upper one is the budgeted run, which stops at 0.0112. The two step curves near the top are ranks, on their own scale: the un-truncated step asks for 5 at every step from the third onwards and the budget allows 2. The truncation is therefore not an occasional tidy-up — it is happening at every step, and the distance between the two residual curves is what it costs. The solution of this problem is itself a train of rank five, so a budget of five or more removes nothing and the two curves coincide.081624324010⁻¹⁴10⁻¹¹10⁻⁸10⁻⁵10⁻²conjugate gradient steprelative residualabove, on their own scale: rank asked 5, rank kept 2solid: the budgeted residual · light: no budgetbudget 2rank asked for5rank kept2residual, budgeted0.011residual, unbudgeted1.6·10⁻¹⁴numbers stored80the step asks for moreat every step

The falling curves are relative residuals: the lower one is the same method with no budget, which reaches 1.61·10⁻¹⁴ in 40 steps, and the upper one is the budgeted run, which stops at 0.0112. The two step curves near the top are ranks, on their own scale: the un-truncated step asks for 5 at every step from the third onwards and the budget allows 2. The truncation is therefore not an occasional tidy-up — it is happening at every step, and the distance between the two residual curves is what it costs. The solution of this problem is itself a train of rank five, so a budget of five or more removes nothing and the two curves coincide.

maxRank: 8

The arguments are the ones An iterate that must be made smaller passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

Conjugate gradients on the three-dimensional model problem with every iterate cut to rank 8The falling curves are relative residuals: the lower one is the same method with no budget, which reaches 1.61·10⁻¹⁴ in 40 steps, and the upper one is the budgeted run, which stops at 1.61·10⁻¹⁴. The two step curves near the top are ranks, on their own scale: the un-truncated step asks for 5 at every step from the third onwards and the budget allows 8. The truncation is therefore not an occasional tidy-up — it is happening at every step, and the distance between the two residual curves is what it costs. The solution of this problem is itself a train of rank five, so a budget of five or more removes nothing and the two curves coincide.081624324010⁻¹⁴10⁻¹¹10⁻⁸10⁻⁵10⁻²conjugate gradient steprelative residualabove, on their own scale: rank asked 5, rank kept 8solid: the budgeted residual · light: no budgetbudget 8rank asked for5rank kept8residual, budgeted1.6·10⁻¹⁴residual, unbudgeted1.6·10⁻¹⁴numbers stored350the step asks for moreat every step

The falling curves are relative residuals: the lower one is the same method with no budget, which reaches 1.61·10⁻¹⁴ in 40 steps, and the upper one is the budgeted run, which stops at 1.61·10⁻¹⁴. The two step curves near the top are ranks, on their own scale: the un-truncated step asks for 5 at every step from the third onwards and the budget allows 8. The truncation is therefore not an occasional tidy-up — it is happening at every step, and the distance between the two residual curves is what it costs. The solution of this problem is itself a train of rank five, so a budget of five or more removes nothing and the two curves coincide.

What it checked while drawing

Every figure above asserted its own claims on the way to being drawn, and a claim that failed would have failed the build rather than drawn a wrong picture. Those assertions used to leave no trace at all: a passing one returned true and the only evidence the figure had checked anything was that nothing crashed. The list below is what they actually said, collected by running this generator with an observer installed — not a description of what it is believed to check.

4 distinct claims across 6 sets of arguments, grouped below by shape — because most of them are one sentence with a different number in it, and how many separate times that sentence was put to the test is the informative part.

a rank budget the solver is given

and the unbudgeted run goes at least as far

matmul shapes agree

the budget is what is kept

Against the rule

It draws a decomposition and prints its residual. It calls truncatedCg, and every figure above carries the badge — which residualcheck verifies by looking for it in the emitted SVG rather than by finding the call that builds one. A badge that is constructed and then left out of the body is the failure that check exists for.

Across the library: the rule bites on 146 of 287 generators — 131 print a residual and 15 are exempt with a published reason; 141 factorise nothing. Read from lib/residual-rule.js, which is the same body the gate enforces from, and the gate's last check fails the build if this page and it disagree about any generator.

Where it is called

Changing this generator changes every figure on this list. That is what makes the list worth publishing rather than keeping in a check script.

Iterating, instead of factorising

An iterate that must be made smaller

Applying a Kronecker-sum operator to a low-rank iterate multiplies its ranks by d and adding two of them adds their ranks, so a solver in a compressed format cannot keep what it produces. Every step is followed by a truncation — and whether that truncation is a floor on the residual depends on the right-hand side rather than on the truncation.

 When the index is a tuple

The format that does not notice the dimension

A Tucker core is r^d numbers, so the format that repaired the definition still cannot go past five indices. Cutting between the indices rather than across them gives d − 1 ranks instead of d, storage linear in the number of indices, and a family whose ranks are two everywhere by an addition formula.

 Neither sparse nor dense

The rounding that was not the problem

A rank-k block plus a rank-k block is a rank-2k block, exactly, so every arithmetic in this format truncates after every addition. A Cholesky performed inside it does ninety-eight of those and its residual is 1.14·10⁻⁹ against a representation error of 1.40·10⁻⁹ — the roundings cost nothing measurable.

The whole library · All essays · What must fail