Numerical Linear Algebra
where the algebra stops being the arithmetic
Generator

picard-plot

One function in the regular library, called 7 times across 4 essays. Below: what it draws at its defaults, what it draws at every value an essay asks for, the 4 claims it put to the test while drawing them, and where it stands against the rule this site is named for.

At its defaults it draws singular values and |uₖᵀb|, with and without 0.10% noise. Three curves against the index on a logarithmic vertical axis. The singular values fall exponentially to the level of rounding. With an exact right-hand side the coefficients fall faster and every term of the solution stays bounded. With noise they flatten at index 32, and from there on each term is noise divided by a σ of 6.8·10⁻⁴.

picard-plot is one function in lib/figures/regular.js — regularisation — the filter, the corner, and the answer that is a choice. Everything below came out of it during this build, at arguments taken from the essays rather than invented for this page. A figure here is the figure a reader meets in an essay, and if the generator changes, this page changes with it.

At its defaults

Drawn even though every essay passes arguments — which on this site is every essay, at 100% of placements since the standard pass. A default nothing exercises is a trap for the next essay to call this with none, and this is the page where a default that has drifted from the figures around it becomes visible.

Singular values and |uₖᵀb|, with and without 0.10% noiseThree curves against the index on a logarithmic vertical axis. The singular values fall exponentially to the level of rounding. With an exact right-hand side the coefficients fall faster and every term of the solution stays bounded. With noise they flatten at index 32, and from there on each term is noise divided by a σ of 6.8·10⁻⁴.081624324048566410⁻¹⁷10⁻¹⁴10⁻¹¹10⁻⁸10⁻⁵10⁻²10¹index kmagnitudethe floor: k = 32best truncation: k = 28σₖ|uₖᵀb| exact|uₖᵀb| with noisetwo different indicesthe crossing, from the data alone32the truncation that is actually best28relative error there0.11the exact coefficients never flattenthe noisy ones stop at ‖e‖/√n

Three curves against the index on a logarithmic vertical axis. The singular values fall exponentially to the level of rounding. With an exact right-hand side the coefficients fall faster and every term of the solution stays bounded. With noise they flatten at index 32, and from there on each term is noise divided by a σ of 6.8·10⁻⁴.

noise: 0.0001

The arguments are the ones Choosing without knowing passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

Singular values and |uₖᵀb|, with and without 0.010% noiseThree curves against the index on a logarithmic vertical axis. The singular values fall exponentially to the level of rounding. With an exact right-hand side the coefficients fall faster and every term of the solution stays bounded. With noise they flatten at index 40, and from there on each term is noise divided by a σ of 1.3·10⁻⁵.081624324048566410⁻¹⁷10⁻¹⁴10⁻¹¹10⁻⁸10⁻⁵10⁻²10¹index kmagnitudethe floor: k = 40best truncation: k = 32σₖ|uₖᵀb| exact|uₖᵀb| with noisetwo different indicesthe crossing, from the data alone40the truncation that is actually best32relative error there0.098the exact coefficients never flattenthe noisy ones stop at ‖e‖/√n

Three curves against the index on a logarithmic vertical axis. The singular values fall exponentially to the level of rounding. With an exact right-hand side the coefficients fall faster and every term of the solution stays bounded. With noise they flatten at index 40, and from there on each term is noise divided by a σ of 1.3·10⁻⁵.

noise: 0.001

The arguments are the ones Rank is a decision passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

Singular values and |uₖᵀb|, with and without 0.10% noiseThree curves against the index on a logarithmic vertical axis. The singular values fall exponentially to the level of rounding. With an exact right-hand side the coefficients fall faster and every term of the solution stays bounded. With noise they flatten at index 32, and from there on each term is noise divided by a σ of 6.8·10⁻⁴.081624324048566410⁻¹⁷10⁻¹⁴10⁻¹¹10⁻⁸10⁻⁵10⁻²10¹index kmagnitudethe floor: k = 32best truncation: k = 28σₖ|uₖᵀb| exact|uₖᵀb| with noisetwo different indicesthe crossing, from the data alone32the truncation that is actually best28relative error there0.11the exact coefficients never flattenthe noisy ones stop at ‖e‖/√n

Three curves against the index on a logarithmic vertical axis. The singular values fall exponentially to the level of rounding. With an exact right-hand side the coefficients fall faster and every term of the solution stays bounded. With noise they flatten at index 32, and from there on each term is noise divided by a σ of 6.8·10⁻⁴.

noise: 0.01

The arguments are the ones Where the answer stops being in the data passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

Singular values and |uₖᵀb|, with and without 1.0% noiseThree curves against the index on a logarithmic vertical axis. The singular values fall exponentially to the level of rounding. With an exact right-hand side the coefficients fall faster and every term of the solution stays bounded. With noise they flatten at index 32, and from there on each term is noise divided by a σ of 6.8·10⁻⁴.081624324048566410⁻¹⁷10⁻¹⁴10⁻¹¹10⁻⁸10⁻⁵10⁻²10¹index kmagnitudethe floor: k = 32best truncation: k = 21σₖ|uₖᵀb| exact|uₖᵀb| with noisetwo different indicesthe crossing, from the data alone32the truncation that is actually best21relative error there0.14the exact coefficients never flattenthe noisy ones stop at ‖e‖/√n

Three curves against the index on a logarithmic vertical axis. The singular values fall exponentially to the level of rounding. With an exact right-hand side the coefficients fall faster and every term of the solution stays bounded. With noise they flatten at index 32, and from there on each term is noise divided by a σ of 6.8·10⁻⁴.

noise: 0.00001

The arguments are the ones Where the answer stops being in the data passes. A value drawn at the generator's defaults instead would be a picture no essay asked for and no assertion has been run against.

Singular values and |uₖᵀb|, with and without 0.0010% noiseThree curves against the index on a logarithmic vertical axis. The singular values fall exponentially to the level of rounding. With an exact right-hand side the coefficients fall faster and every term of the solution stays bounded. With noise they flatten at index 45, and from there on each term is noise divided by a σ of 6.8·10⁻⁷.081624324048566410⁻¹⁷10⁻¹⁴10⁻¹¹10⁻⁸10⁻⁵10⁻²10¹index kmagnitudethe floor: k = 45best truncation: k = 38σₖ|uₖᵀb| exact|uₖᵀb| with noisetwo different indicesthe crossing, from the data alone45the truncation that is actually best38relative error there0.095the exact coefficients never flattenthe noisy ones stop at ‖e‖/√n

Three curves against the index on a logarithmic vertical axis. The singular values fall exponentially to the level of rounding. With an exact right-hand side the coefficients fall faster and every term of the solution stays bounded. With noise they flatten at index 45, and from there on each term is noise divided by a σ of 6.8·10⁻⁷.

What it checked while drawing

Every figure above asserted its own claims on the way to being drawn, and a claim that failed would have failed the build rather than drawn a wrong picture. Those assertions used to leave no trace at all: a passing one returned true and the only evidence the figure had checked anything was that nothing crashed. The list below is what they actually said, collected by running this generator with an observer installed — not a description of what it is believed to check.

4 distinct claims across 5 sets of arguments, grouped below by shape — because most of them are one sentence with a different number in it, and how many separate times that sentence was put to the test is the informative part.

a noise level small enough to be noise

a size the dense SVD is affordable at

and it sits at the noise floor

the crossing is later than the best truncation

Against the rule

The rule does not apply to it. It factorises nothing, so there is no residual it could be withholding. That is worth stating rather than leaving blank: a site that reported the rule as satisfied by every generator would be counting mostly generators the rule never reached.

Across the library: the rule bites on 52 of 99 generators — 37 print a residual and 15 are exempt with a published reason; 47 factorise nothing. Read from lib/residual-rule.js, which is the same body the gate enforces from, and the gate's last check fails the build if this page and it disagree about any generator.

Where it is called

Changing this generator changes every figure on this list. That is what makes the list worth publishing rather than keeping in a check script.

Regularisation, and the answer that is chosen

Choosing without knowing

Three published rules for choosing a regularisation parameter, scored against an oracle that requires the exact answer and is therefore not a method. Generalised cross-validation lands on the oracle's λ exactly; the discrepancy principle costs 6%; the L-curve costs 129%. And told a noise level ten times too small, the discrepancy principle's error goes from 0.112 to 10,449.

 Eigenvalues, singular values, rank

Rank is a decision

A floating-point matrix does not have a rank. It has a spectrum of singular values, and somewhere in that spectrum is a place where the values stop being signal and start being noise. Deciding where is a judgement, and the evidence for it is a gap.

 Regularisation, and the answer that is chosen

When the answer is a choice

A backward-stable least-squares solve of this problem returns an answer whose relative error is 5.5·10⁸. Nothing went wrong. The singular values decay exponentially with no gap anywhere in them, the data does not determine the answer, and something outside the data has to choose — which is the computation rather than a preliminary to it.

 Regularisation, and the answer that is chosen

Where the answer stops being in the data

The Picard condition finds the index where a noisy right-hand side stops carrying signal, from the data alone, with no knowledge of the answer. It lands at 32 where the truncation that actually minimises the error is 28 — and at 45 where the best is 38. It overshoots at every noise level from 1% to 0.001%, and it overshoots for a reason.

The whole library · All essays · What must fail