Generator

A single Newton step solved to eleven inner tolerances, against how far the resulting point is from the root

One function in the sequence library, called 45 times across 7 essays. Below: what it draws at its defaults, what it draws at every value an essay asks for, the 28 claims it put to the test while drawing them, and where it stands against the rule this site is named for.

At its defaults it draws a single newton step solved to eleven inner tolerances, against how far the resulting point is from the root. The iterate is 0.0372 from a root that is known exactly by construction. The same linear system is solved to relative residuals from 0.3 down to 10⁻¹⁴, at 109 and 1126 conjugate gradient iterations, and the resulting point is 0.005319 and 0.002497 from the root. The curve is flat below about 10⁻³: the linearisation is wrong at second order, so the step cannot land closer than the square of the distance it started at — 0.001383 — however exactly it is computed.

inner-plateau is one function in lib/figures/sequence.js — sequences — a solve inside an outer loop, and the accuracy the loop throws away. Everything below came out of it during this build, at arguments taken from the essays rather than invented for this page. A figure here is the figure a reader meets in an essay, and if the generator changes, this page changes with it.

At its defaults

Drawn even though every essay passes arguments — which on this site is every essay, at 100% of placements since the standard pass. A default nothing exercises is a trap for the next essay to call this with none, and this is the page where a default that has drifted from the figures around it becomes visible.

A single Newton step solved to eleven inner tolerances, against how far the resulting point is from the rootThe iterate is 0.0372 from a root that is known exactly by construction. The same linear system is solved to relative residuals from 0.3 down to 10⁻¹⁴, at 109 and 1126 conjugate gradient iterations, and the resulting point is 0.005319 and 0.002497 from the root. The curve is flat below about 10⁻³: the linearisation is wrong at second order, so the step cannot land closer than the square of the distance it started at — 0.001383 — however exactly it is computed.10⁻¹⁵10⁻¹²10⁻⁹10⁻⁶10⁻³110⁻⁴10⁻³10⁻²10⁻¹1inner tolerance η, relative residual of the linear solvedistance from the root after the stepd² = 0.00138distance before the step, 0.03721093547231126the number by each point is the iterations it costeleven decades, one landing placedistance before the step0.037its square0.0014where η = 10⁻³ lands0.0025where η = 10⁻¹⁴ lands0.0025iterations for the first354iterations for the second1126the accuracy that is thrown awaymeasured against a root that is known

The iterate is 0.0372 from a root that is known exactly by construction. The same linear system is solved to relative residuals from 0.3 down to 10⁻¹⁴, at 109 and 1126 conjugate gradient iterations, and the resulting point is 0.005319 and 0.002497 from the root. The curve is flat below about 10⁻³: the linearisation is wrong at second order, so the step cannot land closer than the square of the distance it started at — 0.001383 — however exactly it is computed.

show: "guardc-trade", tol: 1e-8

The arguments are the ones A floor with a cliff at one passes. A value nobody placed would be a picture no essay asked for and no claim was ever checked against.

The last-step floor's constant c, swept, at an outer tolerance of 10⁻⁸: forward error and inner iterations over the unguarded run'sInexact Newton with the adaptive forcing rule on a 200-unknown problem at condition number 10⁵, its inner tolerance floored at c times the outer tolerance times ‖b‖ over ‖F‖. On logarithmic axes against c from 0.01 to 10: the forward error over the unguarded run's, and the inner iterations over its 1109. c = 0.01: error 44×, 983 inner, 10 outer; c = 0.03: error 148×, 932 inner, 10 outer; c = 0.1: error 331×, 897 inner, 10 outer; c = 0.3: error 1677×, 844 inner, 10 outer; c = 0.5: error 2636×, 822 inner, 10 outer; c = 1: error 4979×, 782 inner, 10 outer; c = 3: error 18112×, 774 inner, 79 outer, not converged; c = 10: error 43374×, 733 inner, 79 outer, not converged.tol 10⁻⁸unguarded inner iterations1109c = 1, error ÷ unguarded497910⁻²10⁻¹110¹10⁻¹110¹10²10³10⁴10⁵c, the floor's constantover the unguarded runerror ÷ unguardedinner iterations ÷ unguardedopen dots: the run did not convergea trade with no knee, and then a cliff

Inexact Newton with the adaptive forcing rule on a 200-unknown problem at condition number 10⁵, its inner tolerance floored at c times the outer tolerance times ‖b‖ over ‖F‖. On logarithmic axes against c from 0.01 to 10: the forward error over the unguarded run's, and the inner iterations over its 1109. c = 0.01: error 44×, 983 inner, 10 outer; c = 0.03: error 148×, 932 inner, 10 outer; c = 0.1: error 331×, 897 inner, 10 outer; c = 0.3: error 1677×, 844 inner, 10 outer; c = 0.5: error 2636×, 822 inner, 10 outer; c = 1: error 4979×, 782 inner, 10 outer; c = 3: error 18112×, 774 inner, 79 outer, not converged; c = 10: error 43374×, 733 inner, 79 outer, not converged.

show: "guard"

The arguments are the ones A floor with a cliff at one passes. A value nobody placed would be a picture no essay asked for and no claim was ever checked against.

One line in the adaptive rule: the floor on the last step's forcing termAgainst the outer tolerance on logarithmic axes: the forward error each run reaches, and its total inner iterations drawn on the same axis as a fraction of its range. With the floor the run reaches 1.97·10⁻¹⁰ in 1206 inner iterations at an outer tolerance of 1e-12; without it, 5.25·10⁻¹³ in 1657. Both runs stop when the outer relative residual is below the tolerance, and both do. The floor saves between 9 and 27 per cent of the inner work and costs up to 2636 times the forward error.what one line doesinner work saved, worst case0.09best case0.27forward error cost, worst263610⁻¹²10⁻¹⁰10⁻⁸10⁻⁶10⁻¹²10⁻¹⁰10⁻⁸10⁻⁶10⁻⁴outer toleranceforward errorerror, with the floorerror, without itinner work, rescaledthe grey diagonal is the residual the test asked forboth runs meet it and one is far more accurate

Against the outer tolerance on logarithmic axes: the forward error each run reaches, and its total inner iterations drawn on the same axis as a fraction of its range. With the floor the run reaches 1.97·10⁻¹⁰ in 1206 inner iterations at an outer tolerance of 1e-12; without it, 5.25·10⁻¹³ in 1657. Both runs stop when the outer relative residual is below the tolerance, and both do. The floor saves between 9 and 27 per cent of the inner work and costs up to 2636 times the forward error.

show: "guardc-trade", tol: 1e-12

The arguments are the ones A floor with a cliff at one passes. A value nobody placed would be a picture no essay asked for and no claim was ever checked against.

The last-step floor's constant c, swept, at an outer tolerance of 10⁻¹²: forward error and inner iterations over the unguarded run'sInexact Newton with the adaptive forcing rule on a 200-unknown problem at condition number 10⁵, its inner tolerance floored at c times the outer tolerance times ‖b‖ over ‖F‖. On logarithmic axes against c from 0.01 to 10: the forward error over the unguarded run's, and the inner iterations over its 1657. c = 0.01: error 11×, 1346 inner, 11 outer; c = 0.03: error 42×, 1294 inner, 11 outer; c = 0.1: error 93×, 1262 inner, 11 outer; c = 0.3: error 219×, 1225 inner, 11 outer; c = 0.5: error 376×, 1206 inner, 11 outer; c = 1: error 1005×, 1168 inner, 11 outer; c = 3: error 2229×, 1179 inner, 79 outer, not converged; c = 10: error 9549×, 1139 inner, 79 outer, not converged.tol 10⁻¹²unguarded inner iterations1657c = 1, error ÷ unguarded100510⁻²10⁻¹110¹10⁻¹110¹10²10³10⁴10⁵c, the floor's constantover the unguarded runerror ÷ unguardedinner iterations ÷ unguardedopen dots: the run did not convergea trade with no knee, and then a cliff

Inexact Newton with the adaptive forcing rule on a 200-unknown problem at condition number 10⁵, its inner tolerance floored at c times the outer tolerance times ‖b‖ over ‖F‖. On logarithmic axes against c from 0.01 to 10: the forward error over the unguarded run's, and the inner iterations over its 1657. c = 0.01: error 11×, 1346 inner, 11 outer; c = 0.03: error 42×, 1294 inner, 11 outer; c = 0.1: error 93×, 1262 inner, 11 outer; c = 0.3: error 219×, 1225 inner, 11 outer; c = 0.5: error 376×, 1206 inner, 11 outer; c = 1: error 1005×, 1168 inner, 11 outer; c = 3: error 2229×, 1179 inner, 79 outer, not converged; c = 10: error 9549×, 1139 inner, 79 outer, not converged.

show: "guardc-cliff"

The arguments are the ones A floor with a cliff at one passes. A value nobody placed would be a picture no essay asked for and no claim was ever checked against.

Outer Newton steps against the floor's constant, on five problems at an outer tolerance of 10⁻⁸On logarithmic axes. Each line is one problem, conditioning and nonlinearity as labelled; the outer loop is capped at eighty steps. κ 10⁵, λ 3: 10, 10, 10, 10, 10, 10, 79, 79; κ 10³, λ 3: 8, 8, 8, 8, 8, 8, 22, 79; κ 10⁷, λ 3: 9, 9, 9, 9, 9, 10, 79, 79; κ 10⁵, λ 0.3: 9, 9, 9, 9, 9, 9, 79, 79; κ 10⁵, λ 30: 9, 9, 9, 9, 9, 9, 79, 79 for c = 0.01, 0.03, 0.1, 0.3, 0.5, 1, 3, 10. Every problem converges in its usual count up to c = 1 and stalls at the cap from c = 3 or 10.10⁻²10⁻¹110¹10¹10²c, the floor's constantouter Newton stepsκ 10⁵, λ 3κ 10³, λ 3κ 10⁷, λ 3κ 10⁵, λ 0.3κ 10⁵, λ 30eighty is the cap: the run stalleda floor above the test is a floor the test cannot pass

On logarithmic axes. Each line is one problem, conditioning and nonlinearity as labelled; the outer loop is capped at eighty steps. κ 10⁵, λ 3: 10, 10, 10, 10, 10, 10, 79, 79; κ 10³, λ 3: 8, 8, 8, 8, 8, 8, 22, 79; κ 10⁷, λ 3: 9, 9, 9, 9, 9, 10, 79, 79; κ 10⁵, λ 0.3: 9, 9, 9, 9, 9, 9, 79, 79; κ 10⁵, λ 30: 9, 9, 9, 9, 9, 9, 79, 79 for c = 0.01, 0.03, 0.1, 0.3, 0.5, 1, 3, 10. Every problem converges in its usual count up to c = 1 and stalls at the cap from c = 3 or 10.

show: "guardc-premium"

The arguments are the ones A floor with a cliff at one passes. A value nobody placed would be a picture no essay asked for and no claim was ever checked against.

What the floor at c = ½ saves and what it costs, on five problems at four outer tolerancesTwenty runs: five problems, conditioning 10³ to 10⁷ and nonlinearity 0.3 to 30, at outer tolerances 10⁻⁶ to 10⁻¹². Horizontally, the share of inner iterations the floor saves; vertically, on a logarithmic axis, the forward error with the floor over the error without it. The premium runs from 1.00 to 4.81·10⁴ and the saving from 0.0% to 37.7%, with no relation between them: κ 10⁵, λ 3 at 10⁻⁶, 19.6% and 31.7; κ 10⁵, λ 3 at 10⁻⁸, 25.9% and 2636; κ 10⁵, λ 3 at 10⁻¹⁰, 9.0% and 23.2; κ 10⁵, λ 3 at 10⁻¹², 27.2% and 376; κ 10³, λ 3 at 10⁻⁶, 32.5% and 1221; κ 10³, λ 3 at 10⁻⁸, 11.3% and 12.6; κ 10³, λ 3 at 10⁻¹⁰, 31.7% and 4.81·10⁴; κ 10³, λ 3 at 10⁻¹², 18.9% and 587; κ 10⁷, λ 3 at 10⁻⁶, 37.7% and 1.02; κ 10⁷, λ 3 at 10⁻⁸, 0.0% and 1.00; κ 10⁷, λ 3 at 10⁻¹⁰, 2.4% and 1.74; κ 10⁷, λ 3 at 10⁻¹², 17.5% and 300; κ 10⁵, λ 0.3 at 10⁻⁶, 24.5% and 3.70; κ 10⁵, λ 0.3 at 10⁻⁸, 20.3% and 131; κ 10⁵, λ 0.3 at 10⁻¹⁰, 2.4% and 1.69; κ 10⁵, λ 0.3 at 10⁻¹², 19.8% and 374; κ 10⁵, λ 30 at 10⁻⁶, 28.4% and 374; κ 10⁵, λ 30 at 10⁻⁸, 5.5% and 2.92; κ 10⁵, λ 30 at 10⁻¹⁰, 23.8% and 3935; κ 10⁵, λ 30 at 10⁻¹², 10.5% and 44.4.error with floor ÷ withoutsmallest premium1largest premium4.8·10⁴010203040110¹10²10³10⁴10⁵inner iterations saved, per centerror with floor ÷ withoutκ 10⁵, λ 3κ 10³, λ 3κ 10⁷, λ 3κ 10⁵, λ 0.3κ 10⁵, λ 30each dot one problem at one tolerancethe price of the saving is not predictable

Twenty runs: five problems, conditioning 10³ to 10⁷ and nonlinearity 0.3 to 30, at outer tolerances 10⁻⁶ to 10⁻¹². Horizontally, the share of inner iterations the floor saves; vertically, on a logarithmic axis, the forward error with the floor over the error without it. The premium runs from 1.00 to 4.81·10⁴ and the saving from 0.0% to 37.7%, with no relation between them: κ 10⁵, λ 3 at 10⁻⁶, 19.6% and 31.7; κ 10⁵, λ 3 at 10⁻⁸, 25.9% and 2636; κ 10⁵, λ 3 at 10⁻¹⁰, 9.0% and 23.2; κ 10⁵, λ 3 at 10⁻¹², 27.2% and 376; κ 10³, λ 3 at 10⁻⁶, 32.5% and 1221; κ 10³, λ 3 at 10⁻⁸, 11.3% and 12.6; κ 10³, λ 3 at 10⁻¹⁰, 31.7% and 4.81·10⁴; κ 10³, λ 3 at 10⁻¹², 18.9% and 587; κ 10⁷, λ 3 at 10⁻⁶, 37.7% and 1.02; κ 10⁷, λ 3 at 10⁻⁸, 0.0% and 1.00; κ 10⁷, λ 3 at 10⁻¹⁰, 2.4% and 1.74; κ 10⁷, λ 3 at 10⁻¹², 17.5% and 300; κ 10⁵, λ 0.3 at 10⁻⁶, 24.5% and 3.70; κ 10⁵, λ 0.3 at 10⁻⁸, 20.3% and 131; κ 10⁵, λ 0.3 at 10⁻¹⁰, 2.4% and 1.69; κ 10⁵, λ 0.3 at 10⁻¹², 19.8% and 374; κ 10⁵, λ 30 at 10⁻⁶, 28.4% and 374; κ 10⁵, λ 30 at 10⁻⁸, 5.5% and 2.92; κ 10⁵, λ 30 at 10⁻¹⁰, 23.8% and 3935; κ 10⁵, λ 30 at 10⁻¹², 10.5% and 44.4.

What it checked while drawing

Every figure above checked its own claims on the way to being drawn, and a claim that failed would have stopped the picture rather than shipped a wrong one. Those checks used to leave no trace at all: a passing one returned true and the only evidence the figure had checked anything was that nothing crashed. The list below is what they actually said, collected by running this generator with an observer installed — not a description of what it is believed to check.

28 distinct claims across 6 sets of arguments, grouped below by shape — because most of them are one sentence with a different number in it, and how many separate times that sentence was put to the test is the informative part.

the guard never costs inner iterations at tol = 0.000001 — checked 4 times

a conditioning the SPD construction can hold

a problem the forcing essays define

a problem the forcing sweep runs

a problem the forward sweep runs

a problem the guard sweep defines

a size the dense factorisations are affordable at

a stopping rule this file defines

a warm start this routine implements

an inner-solve view this figure draws

an iterate the exact Newton sequence reaches

an outer tolerance that is a relative residual

an outer tolerance the constant sweep measures

and a tolerance of 10⁻³ reaches the same point as one of 10⁻¹⁴

and both runs pass the test they were given

and somewhere it costs an order of magnitude of forward error

and the scaled one is better at a tight forcing term

enough tolerances to see a plateau rather than two points

LU is for square matrices

matmul shapes agree

one of Eisenstat and Walker's two forcing choices

the exact Newton step exists at every iterate walked to

the known rule is told κ(A)

the tightest tolerance costs the most iterations

the unscaled previous step is worse than no guess

Against the rule

The rule does not apply to it. It factorises nothing, so there is no residual it could be withholding. That is worth stating rather than leaving blank: a site that reported the rule as satisfied by every generator would be counting mostly generators the rule never reached.

Across the library: the rule bites on 217 of 397 generators — 199 print a residual and 18 are exempt with a published reason; 180 factorise nothing. Read from lib/residual-rule.js, which is the same body the gate enforces from, and the gate's last check fails the build if this page and it disagree about any generator.

Where it is called

Changing this generator changes every figure on this list. That is what makes the list worth publishing rather than keeping in a check script.

When the problem arrives again

A floor with a cliff at one

Inexact Newton's adaptive forcing rule is used with a floor: never ask the inner solve for less than c times the outer tolerance, relative to the current residual, with c a half. Swept from a hundredth to ten, the constant trades with no knee at all — the forward error rises in proportion to c, the inner work falls by a few per cent a decade — and then, just past one, the run stops converging: at c = 3 it hits the outer limit on four problems of five, having asked every step for less than the stopping test needs. And the accuracy the floor gives up at c = ½ is no planning number: across five problems and four tolerances it runs from nothing to a factor of 48,000.

When the problem arrives again

A guess worth two per cent

The previous Newton step looks like a free guess at the next one, and it is worth nothing. Started from it unscaled, the inner solve costs 4 to 61 per cent more than starting from zero, because the guess is 15 to 209 times too large. Scaled by the ratio of the two residual norms it is the right size and halves the starting residual — which buys a constant handful of inner iterations, not a share, because conjugate gradients costs the logarithm of its tolerance.

When the problem arrives again

A target the residual can promise

A caller who wants inexact Newton's answer to six digits usually passes 10⁻⁶ as the residual tolerance, and on five problems at four targets that misses by 7 to 35,000 times. Divide the target by a condition number and it is met on every cell where a residual can certify it — and the condition number need not be known: the Ritz values conjugate gradients produces on the way give the Jacobian's own, settled by the sixth Newton step and within eleven per cent of the truth at the end, and two to nineteen times below the linear part's. Stopped that way, with the usual floor under the forcing term, the run lands within a fifth of its target and is cheaper on 16 of 19 cells than the unfloored run stopped by someone who knows the answer.

When the problem arrives again

A tolerance that reads its own residual

The cheapest constant forcing term costs 980 inner iterations and arrives with a hundred times the forward error of the dearest, which costs 9,358. A rule that sets each step's tolerance from the ratio of the last two residuals costs 1,009 and arrives with neither problem — and it is not a constant, so it does not appear on the curve the constants are compared on.

When the problem arrives again

One line that buys a quarter of the run

The adaptive forcing rule has a floor on it that no published statement of the rule carries: do not solve a step to an accuracy the outer loop will not use. Removing it costs 9 to 27 per cent of the whole inner run. Keeping it costs between 23 and 2,600 times the forward error — accuracy the residual test never asked for and both runs satisfy the test either way. The line is a trade between a residual and an error, and which of the two the caller meant decides whether it is a saving.

When the problem arrives again

The accuracy that is thrown away

A Newton step is the exact answer to a linearised problem, and the linearisation is wrong at second order. So there is a floor under how close the step can land, the floor is the square of where it started, and eleven decades of inner tolerance below it buy the same four digits at four times the price.

When the problem arrives again

Two forcing terms, one floor

Inexact Newton's adaptive forcing rule comes in two versions, and every measurement of the floor under it used the second, which reads the ratio of the last two residuals. The first reads how wrong the linear model was at the last step, and it was natural to hope that a rule watching the model's own error would not solve its last step past what the stopping test needs. It does. Run without the floor on five problems at four forward targets, it over-solves its last step by a median of 34 times, against the second rule's 95; the floor saves 17.5 per cent of its inner work, against 16.0. Floored, the first rule is the cheaper by 4.6 per cent overall, and it buys that with Newton steps: two more on the median cell, a third less inner work on the strongly nonlinear problem, and 4.5 per cent more on the easy one.

The whole library · All essays · What must fail