A single Newton step solved to eleven inner tolerances, against how far the resulting point is from the root
At its defaults it draws a single newton step solved to eleven inner tolerances, against how far the resulting point is from the root. The iterate is 0.0372 from a root that is known exactly by construction. The same linear system is solved to relative residuals from 0.3 down to 10⁻¹⁴, at 109 and 1126 conjugate gradient iterations, and the resulting point is 0.005319 and 0.002497 from the root. The curve is flat below about 10⁻³: the linearisation is wrong at second order, so the step cannot land closer than the square of the distance it started at — 0.001383 — however exactly it is computed.
inner-plateau is one function in lib/figures/sequence.js —
sequences — a solve inside an outer loop, and the accuracy the loop throws away. Everything below came out of it during this build, at
arguments taken from the essays rather than invented for this page. A figure here is the
figure a reader meets in an essay, and if the generator changes, this page changes with it.
At its defaults
Drawn even though every essay passes arguments — which on this site is every essay, at 100% of placements since the standard pass. A default nothing exercises is a trap for the next essay to call this with none, and this is the page where a default that has drifted from the figures around it becomes visible.
The iterate is 0.0372 from a root that is known exactly by construction. The same linear system is solved to relative residuals from 0.3 down to 10⁻¹⁴, at 109 and 1126 conjugate gradient iterations, and the resulting point is 0.005319 and 0.002497 from the root. The curve is flat below about 10⁻³: the linearisation is wrong at second order, so the step cannot land closer than the square of the distance it started at — 0.001383 — however exactly it is computed.
show: "guardc-trade", tol: 1e-8
The arguments are the ones A floor with a cliff at one passes. A value nobody placed would be a picture no essay asked for and no claim was ever checked against.
Inexact Newton with the adaptive forcing rule on a 200-unknown problem at condition number 10⁵, its inner tolerance floored at c times the outer tolerance times ‖b‖ over ‖F‖. On logarithmic axes against c from 0.01 to 10: the forward error over the unguarded run's, and the inner iterations over its 1109. c = 0.01: error 44×, 983 inner, 10 outer; c = 0.03: error 148×, 932 inner, 10 outer; c = 0.1: error 331×, 897 inner, 10 outer; c = 0.3: error 1677×, 844 inner, 10 outer; c = 0.5: error 2636×, 822 inner, 10 outer; c = 1: error 4979×, 782 inner, 10 outer; c = 3: error 18112×, 774 inner, 79 outer, not converged; c = 10: error 43374×, 733 inner, 79 outer, not converged.
show: "guard"
The arguments are the ones A floor with a cliff at one passes. A value nobody placed would be a picture no essay asked for and no claim was ever checked against.
Against the outer tolerance on logarithmic axes: the forward error each run reaches, and its total inner iterations drawn on the same axis as a fraction of its range. With the floor the run reaches 1.97·10⁻¹⁰ in 1206 inner iterations at an outer tolerance of 1e-12; without it, 5.25·10⁻¹³ in 1657. Both runs stop when the outer relative residual is below the tolerance, and both do. The floor saves between 9 and 27 per cent of the inner work and costs up to 2636 times the forward error.
show: "guardc-trade", tol: 1e-12
The arguments are the ones A floor with a cliff at one passes. A value nobody placed would be a picture no essay asked for and no claim was ever checked against.
Inexact Newton with the adaptive forcing rule on a 200-unknown problem at condition number 10⁵, its inner tolerance floored at c times the outer tolerance times ‖b‖ over ‖F‖. On logarithmic axes against c from 0.01 to 10: the forward error over the unguarded run's, and the inner iterations over its 1657. c = 0.01: error 11×, 1346 inner, 11 outer; c = 0.03: error 42×, 1294 inner, 11 outer; c = 0.1: error 93×, 1262 inner, 11 outer; c = 0.3: error 219×, 1225 inner, 11 outer; c = 0.5: error 376×, 1206 inner, 11 outer; c = 1: error 1005×, 1168 inner, 11 outer; c = 3: error 2229×, 1179 inner, 79 outer, not converged; c = 10: error 9549×, 1139 inner, 79 outer, not converged.
show: "guardc-cliff"
The arguments are the ones A floor with a cliff at one passes. A value nobody placed would be a picture no essay asked for and no claim was ever checked against.
On logarithmic axes. Each line is one problem, conditioning and nonlinearity as labelled; the outer loop is capped at eighty steps. κ 10⁵, λ 3: 10, 10, 10, 10, 10, 10, 79, 79; κ 10³, λ 3: 8, 8, 8, 8, 8, 8, 22, 79; κ 10⁷, λ 3: 9, 9, 9, 9, 9, 10, 79, 79; κ 10⁵, λ 0.3: 9, 9, 9, 9, 9, 9, 79, 79; κ 10⁵, λ 30: 9, 9, 9, 9, 9, 9, 79, 79 for c = 0.01, 0.03, 0.1, 0.3, 0.5, 1, 3, 10. Every problem converges in its usual count up to c = 1 and stalls at the cap from c = 3 or 10.
show: "guardc-premium"
The arguments are the ones A floor with a cliff at one passes. A value nobody placed would be a picture no essay asked for and no claim was ever checked against.
Twenty runs: five problems, conditioning 10³ to 10⁷ and nonlinearity 0.3 to 30, at outer tolerances 10⁻⁶ to 10⁻¹². Horizontally, the share of inner iterations the floor saves; vertically, on a logarithmic axis, the forward error with the floor over the error without it. The premium runs from 1.00 to 4.81·10⁴ and the saving from 0.0% to 37.7%, with no relation between them: κ 10⁵, λ 3 at 10⁻⁶, 19.6% and 31.7; κ 10⁵, λ 3 at 10⁻⁸, 25.9% and 2636; κ 10⁵, λ 3 at 10⁻¹⁰, 9.0% and 23.2; κ 10⁵, λ 3 at 10⁻¹², 27.2% and 376; κ 10³, λ 3 at 10⁻⁶, 32.5% and 1221; κ 10³, λ 3 at 10⁻⁸, 11.3% and 12.6; κ 10³, λ 3 at 10⁻¹⁰, 31.7% and 4.81·10⁴; κ 10³, λ 3 at 10⁻¹², 18.9% and 587; κ 10⁷, λ 3 at 10⁻⁶, 37.7% and 1.02; κ 10⁷, λ 3 at 10⁻⁸, 0.0% and 1.00; κ 10⁷, λ 3 at 10⁻¹⁰, 2.4% and 1.74; κ 10⁷, λ 3 at 10⁻¹², 17.5% and 300; κ 10⁵, λ 0.3 at 10⁻⁶, 24.5% and 3.70; κ 10⁵, λ 0.3 at 10⁻⁸, 20.3% and 131; κ 10⁵, λ 0.3 at 10⁻¹⁰, 2.4% and 1.69; κ 10⁵, λ 0.3 at 10⁻¹², 19.8% and 374; κ 10⁵, λ 30 at 10⁻⁶, 28.4% and 374; κ 10⁵, λ 30 at 10⁻⁸, 5.5% and 2.92; κ 10⁵, λ 30 at 10⁻¹⁰, 23.8% and 3935; κ 10⁵, λ 30 at 10⁻¹², 10.5% and 44.4.
What it checked while drawing
Every figure above checked its own claims on the way to being drawn, and a claim that failed
would have stopped the picture rather than shipped a wrong one. Those checks used to leave
no trace at all: a passing one returned true and the only evidence the figure had
checked anything was that nothing crashed. The list below is what they actually said, collected
by running this generator with an observer installed — not a description of
what it is believed to check.
28 distinct claims across 6 sets of arguments, grouped below by shape — because most of them are one sentence with a different number in it, and how many separate times that sentence was put to the test is the informative part.
the guard never costs inner iterations at tol = 0.000001 — checked 4 times
a conditioning the SPD construction can hold
a problem the forcing essays define
a problem the forcing sweep runs
a problem the forward sweep runs
a problem the guard sweep defines
a size the dense factorisations are affordable at
a stopping rule this file defines
a warm start this routine implements
an inner-solve view this figure draws
an iterate the exact Newton sequence reaches
an outer tolerance that is a relative residual
an outer tolerance the constant sweep measures
and a tolerance of 10⁻³ reaches the same point as one of 10⁻¹⁴
and both runs pass the test they were given
and somewhere it costs an order of magnitude of forward error
and the scaled one is better at a tight forcing term
enough tolerances to see a plateau rather than two points
LU is for square matrices
matmul shapes agree
one of Eisenstat and Walker's two forcing choices
the exact Newton step exists at every iterate walked to
the known rule is told κ(A)
the tightest tolerance costs the most iterations
the unscaled previous step is worse than no guess
Against the rule
The rule does not apply to it. It factorises nothing, so there is no residual it could be withholding. That is worth stating rather than leaving blank: a site that reported the rule as satisfied by every generator would be counting mostly generators the rule never reached.
Across the library: the rule bites on 217
of 397 generators —
199 print a residual and
18 are exempt with a published reason;
180 factorise nothing.
Read from lib/residual-rule.js, which is the same body the gate enforces from,
and the gate's last check fails the build if this page and it disagree about any generator.
Where it is called
Changing this generator changes every figure on this list. That is what makes the list worth publishing rather than keeping in a check script.
A floor with a cliff at one
Inexact Newton's adaptive forcing rule is used with a floor: never ask the inner solve for less than c times the outer tolerance, relative to the current residual, with c a half. Swept from a hundredth to ten, the constant trades with no knee at all — the forward error rises in proportion to c, the inner work falls by a few per cent a decade — and then, just past one, the run stops converging: at c = 3 it hits the outer limit on four problems of five, having asked every step for less than the stopping test needs. And the accuracy the floor gives up at c = ½ is no planning number: across five problems and four tolerances it runs from nothing to a factor of 48,000.
When the problem arrives againA guess worth two per cent
The previous Newton step looks like a free guess at the next one, and it is worth nothing. Started from it unscaled, the inner solve costs 4 to 61 per cent more than starting from zero, because the guess is 15 to 209 times too large. Scaled by the ratio of the two residual norms it is the right size and halves the starting residual — which buys a constant handful of inner iterations, not a share, because conjugate gradients costs the logarithm of its tolerance.
When the problem arrives againA target the residual can promise
A caller who wants inexact Newton's answer to six digits usually passes 10⁻⁶ as the residual tolerance, and on five problems at four targets that misses by 7 to 35,000 times. Divide the target by a condition number and it is met on every cell where a residual can certify it — and the condition number need not be known: the Ritz values conjugate gradients produces on the way give the Jacobian's own, settled by the sixth Newton step and within eleven per cent of the truth at the end, and two to nineteen times below the linear part's. Stopped that way, with the usual floor under the forcing term, the run lands within a fifth of its target and is cheaper on 16 of 19 cells than the unfloored run stopped by someone who knows the answer.
When the problem arrives againA tolerance that reads its own residual
The cheapest constant forcing term costs 980 inner iterations and arrives with a hundred times the forward error of the dearest, which costs 9,358. A rule that sets each step's tolerance from the ratio of the last two residuals costs 1,009 and arrives with neither problem — and it is not a constant, so it does not appear on the curve the constants are compared on.
When the problem arrives againOne line that buys a quarter of the run
The adaptive forcing rule has a floor on it that no published statement of the rule carries: do not solve a step to an accuracy the outer loop will not use. Removing it costs 9 to 27 per cent of the whole inner run. Keeping it costs between 23 and 2,600 times the forward error — accuracy the residual test never asked for and both runs satisfy the test either way. The line is a trade between a residual and an error, and which of the two the caller meant decides whether it is a saving.
When the problem arrives againThe accuracy that is thrown away
A Newton step is the exact answer to a linearised problem, and the linearisation is wrong at second order. So there is a floor under how close the step can land, the floor is the square of where it started, and eleven decades of inner tolerance below it buy the same four digits at four times the price.
When the problem arrives againTwo forcing terms, one floor
Inexact Newton's adaptive forcing rule comes in two versions, and every measurement of the floor under it used the second, which reads the ratio of the last two residuals. The first reads how wrong the linear model was at the last step, and it was natural to hope that a rule watching the model's own error would not solve its last step past what the stopping test needs. It does. Run without the floor on five problems at four forward targets, it over-solves its last step by a median of 34 times, against the second rule's 95; the floor saves 17.5 per cent of its inner work, against 16.0. Floored, the first rule is the cheaper by 4.6 per cent overall, and it buys that with Newton steps: two more on the median cell, a third less inner work on the strongly nonlinear problem, and 4.5 per cent more on the easy one.