Numerical Linear Algebra
where the algebra stops being the arithmetic
When the problem arrives again

The gap was the square

Eisenstat and Walker's second forcing rule was a third dearer than the first on a strongly nonlinear problem, and the cost had been traced to one over-tight step: its safeguard is skipped below 0.1, so the forcing term fell to 0.024 and the step took 109 inner iterations. Applying the safeguard always was predicted to hold that step at 0.04, halve its cost and bring the rule within fifteen per cent of the first. It holds the step at 0.038 and saves twenty iterations; the rule stays forty per cent dearer. The residual still rises after that step, because at that point Newton's own step overshoots and any solve tighter than about 0.06 reproduces it. A line search removes the rise and the reset and still leaves the second rule thirty-six per cent above the first. What is left is the rule itself: 0.9 times the square of the last residual ratio, which after every fast fall asks for a solve far tighter than the next step needs.
20 min read 5 figures Measured, not assumedWhose fault is itClaims that can fail

Worth reading first: The accuracy that is thrown away · A parameter that counts steps.

The step before the reset went looking for why Eisenstat and Walker’s second forcing rule costs a third more inner iterations than their first on a strongly nonlinear problem, and found the cost in an unexpected place. The obvious suspect was the reset: when the residual rose, the second rule’s forcing term jumped back to 0.9 and the method took several loose, cheap steps to tighten again. Capping that jump removed two Newton steps and saved 0.7 per cent of the work. The cost was the step before the rise, where the rule’s forcing term had fallen to 0.024 and the step took 109 inner iterations, where the first rule’s matching step took five.

It also found why the forcing term fell so far. The second rule sets ηk=0.9 (∥Fk∥/∥Fk−1∥)2\eta_k = 0.9\,(\|F_k\|/\|F_{k-1}\|)^2, and to stop it falling too fast it is held at or above 0.9 ηk−120.9\,\eta_{k-1}^2 — but only when that safeguard exceeds 0.1. At the fifth step the safeguard was 0.9×0.2062=0.0380.9 \times 0.206^2 = 0.038, under the threshold, so it was skipped. The essay closed on the repair this suggested: “lowering the threshold to zero — applying the safeguard always — holds the fifth step’s forcing term at or above 0.04, costs it fewer than half the 109 iterations it spent, and brings the second rule’s work on that problem within fifteen per cent of the first rule’s, while costing under two per cent on the other four.”

One of those four numbers survives.

The safeguard, always

The five problems are the ones every essay on forcing terms here has used — 200 unknowns, the Jacobian’s condition number 10310^3 to 10710^7, the nonlinearity’s strength λ\lambda from 0.3 to 30 — and the stopping rule and the floor on the forcing term are the ones two forcing terms, one floor settled on. The second rule is run four ways: with its safeguard skipped below 0.1, as Eisenstat and Walker wrote it; skipped below 0.03, below 0.01, and never.

Inner iterations spent so far at each Newton step on the strongly nonlinear problem: the first forcing rule, the second, the second with its safeguard always applied, and the second with a backtracking searchForward target ten to the minus four, all floored. first rule: 1, 2, 3, 5, 10, 33, 71, 162, 253, 337; second rule: 1, 2, 4, 13, 122, 128, 136, 151, 177, 263, 437, 542; second, guard always: 1, 2, 4, 13, 102, 108, 118, 125, 154, 234, 391, 505; second, damped: 1, 2, 4, 13, 122, 147, 216, 367, 484.inner iterations in allfirst rule337second rule542second, guard always505second, damped4841234567891011120100200300400500600Newton stepinner iterations so farfirst rulesecond rulesecond, guard alwayssecond, dampedstrong problem, target 10⁻⁴the gap survives every repair
Fig. 1 Inner iterations spent so far at each Newton step on the strongly nonlinear problem, at a forward target of 10−410^{-4}: the first rule, the second, the second with its safeguard always applied, and the second with a backtracking search.

The three lower thresholds give the same run, iteration for iteration, on every problem at every target: whenever the safeguard is consulted at all on these problems, it is above 0.03. With it always applied, the fifth step on the strong problem is solved to 0.038 rather than 0.024 and costs 89 inner iterations rather than 109. That is the safeguard’s own value — the prediction’s “at or above 0.04” was the same number rounded up — and twenty iterations saved, not fifty-five.

Over the four forward targets the strong problem can certify, the second rule’s work falls by four per cent, from 1.46 times the first rule’s to 1.40. On the other four problems the change costs at most 0.7 per cent and on one of them saves 4.8 — that part of the prediction holds. But the gap the repair was for is nearly all still there, and the figure shows where it is: the always-guarded second rule’s residual rises after the fifth step exactly as the default’s did, the rule resets to 0.9, and it spends the next seven steps climbing back.

Newton’s own step overshoots there

So the rise is not the over-tight solve’s fault either, or not only. The direct test is to stand where the second rule stands before its fifth step — the same point, the same Jacobian, the same residual — and solve that one Newton system to every forcing term on a grid, from 0.9 down to 10−410^{-4}.

At the Newton step before the second forcing rule's residual rises on the strongly nonlinear problem, the residual a step leaves over the one it started from, for forcing terms from ten to the minus four to 0.9forcing term 0.9: 11 inner iterations, step length 4.7, residual ratio 0.732; forcing term 0.7: 12 inner iterations, step length 4.9, residual ratio 0.604; forcing term 0.5: 21 inner iterations, step length 7.8, residual ratio 0.529; forcing term 0.35: 23 inner iterations, step length 8.3, residual ratio 0.436; forcing term 0.25: 36 inner iterations, step length 11.2, residual ratio 0.532; forcing term 0.18: 39 inner iterations, step length 11.6, residual ratio 0.487; forcing term 0.12: 59 inner iterations, step length 14.8, residual ratio 0.823; forcing term 0.08: 66 inner iterations, step length 15.5, residual ratio 0.885; forcing term 0.05: 80 inner iterations, step length 16.4, residual ratio 1.017; forcing term 0.038: 89 inner iterations, step length 16.7, residual ratio 1.061; forcing term 0.02: 109 inner iterations, step length 17.1, residual ratio 1.092; forcing term 0.01: 124 inner iterations, step length 17.2, residual ratio 1.099; forcing term 0.001: 187 inner iterations, step length 17.3, residual ratio 1.100; forcing term 0.0001: 243 inner iterations, step length 17.3, residual ratio 1.100. Every forcing term down to 0.08 lowers the residual; every one from 0.05 down raises it.10⁻⁴10⁻³10⁻²10⁻¹10.30.40.50.60.70.80.911.11.2forcing term of the stepresidual after over beforesecond ruleguard alwaysno changethe first rule took 0.31 at this pointa loose step is a short step
Fig. 2 At the step before the rise, the residual one step leaves over the one it started from, for each forcing term the inner solve is stopped at. The dashed line is no change; the dotted lines are the forcing terms the second rule chose with and without its safeguard.

Every forcing term from 0.08 up lowers the residual. Every one from 0.05 down raises it, and the solves tighter than 10−210^{-2} all raise it by ten per cent: they are all close to the exact Newton step, and the exact Newton step from this point overshoots. The best single step in the grid is at a forcing term of 0.35, which cuts the residual to 0.44 of what it was. The first rule, at its own comparable step a little further on, chose 0.31.

The mechanism is a property of conjugate gradients that every inexact Newton method inherits whether it wants it or not: from a zero start, the length of the conjugate-gradient iterate grows monotonically with the iterations. At this point the step is 4.7 long at a forcing term of 0.9, 8.3 at 0.35, 16.7 at the second rule’s 0.038 and 17.3 at the exact solution. A loose solve is a short step, and a short step taken where the full one overshoots is a damped step — the line search the method does not otherwise have. A Krylov space for a problem that is not linear traced how much of Newton’s model a few Krylov steps keep; here keeping less of it is what keeps the residual falling.

Priced in inner iterations the loose end of the scan wins outright. A forcing term of 0.9 costs 11 iterations and leaves 0.73 of the residual; 0.35 costs 23 and leaves 0.44; 0.08 costs 66 and leaves 0.89; the second rule’s 0.024 cost 109 and left 1.10. Past the valley’s floor every extra iteration buys a longer step in a direction the nonlinearity has already stopped honouring.

The second rule, by asking for a tight solve at exactly the point where the model is least trustworthy, buys the overshoot. Its safeguard always applied asks for a slightly less tight one — still well past the turning point near 0.06 — and buys it too.

Remove the rise and the gap stays

If the rise and the reset were what made the second rule dear, removing them should close the gap. A backtracking search does that directly: after each inner solve, halve the step until it does not raise the residual. Nothing else changes — the same forcing rules, the same floor, the same stopping test.

On the strong problem the second rule needs one halving a run, at the fifth step, and the first rule never needs one. Damped, the second rule’s residual never rises, its forcing term never resets, and it converges in three fewer Newton steps than undamped — one fewer than the first rule. The halved fifth step leaves the residual at half of where it started, the rule reads a ratio of one half, and it asks the sixth step for 0.215: the same loose step the undamped rule only reached by resetting to 0.9 and climbing down through three more Newton steps. Damping turns the reset’s four steps into one. Its work falls by seven per cent. And summed over the four targets it is still 1.36 times the first rule’s.

The forcing term each Newton step was solved to on the strongly nonlinear problem, the first rule against the second, dampedForward target ten to the minus four. first rule: 0.900 (1), 0.843 (1), 0.759 (1), 0.640 (2), 0.486 (5), 0.311 (23), 0.151 (38), 0.018 (91), 0.026 (91), 0.055 (84); second, damped: 0.900 (1), 0.729 (1), 0.478 (2), 0.206 (9), 0.024 (109), 0.215 (25), 0.047 (69), 0.003 (151), 0.023 (117). Inner iterations in brackets.inner iterations in allfirst rule, 10 steps337second, damped, 9 steps4841234567891011121310⁻⁴10⁻³10⁻²10⁻¹1Newton stepforcing termfirst rulesecond, dampedstrong problem, target 10⁻⁴the square tightens too fast
Fig. 3 The forcing term each Newton step was solved to on the strong problem at a target of 10−410^{-4}, the first rule against one variant of the second, with each run’s total. The dial sets the variant: default, safeguard always, damped.

Set the dial to the damped run and the reset is gone — the second rule’s forcing term never returns to 0.9 — and the excess is in plain view on two steps. The fifth is still solved to 0.024, costing 109 inner iterations, because a backtracking search acts after the solve and cannot make it cheaper. The eighth is solved to 0.003 at 151 iterations, where the first rule’s matching step takes 0.018 and 91. Of the damped second rule’s 484 inner iterations at this target, those two steps take 260; the first rule’s two tightest steps take 182 of its 337. The excess of 147 is split almost evenly between the two over-tight steps and the rest of the run, where the second rule’s forcing terms sit a little below the first’s at almost every step.

Target by target the picture is the same and the gap narrows as the target tightens, because the late steps, where the two rules behave alike, are a growing share of the run. Over the first rule’s work the always-guarded second rule costs 1.50, 1.44, 1.39 and 1.34 at targets of 10−410^{-4}, 10−610^{-6}, 10−810^{-8} and 10−1010^{-10}; damped, 1.44, 1.38, 1.35 and 1.30. Against the default second rule’s 1.61, 1.49, 1.43 and 1.39, a line search buys eight to seventeen points at each target and the safeguard always applied four to eleven; neither closes more than three tenths of the gap at any target.

A fast fall buys a tight solve

Both over-tight steps follow a fast fall in the residual. The second rule’s forcing term is 0.9 r20.9\,r^2 with rr the ratio of the last two residuals, so a step that cuts the residual by a factor of six — ratio 0.16 — buys the next step a forcing term of 0.024, and one that cuts it by seventeen buys 0.003. The rule was designed for the end of a Newton iteration, where a fast fall means quadratic convergence has begun and the next step’s model is good enough to solve accurately. In the middle of one, on a strongly nonlinear problem, a fast fall is often one good step between two bad ones.

The forcing term each damped Newton step was solved to against the ratio of the last two residuals, first and second rule, on the strongly nonlinear problem at every target, with the second rule's formula, 0.9 times the ratio squaredSecond rule: 0.364 to 0.7290, 0.613 to 0.4783, 0.369 to 0.2059, 0.162 to 0.0235, 0.489 to 0.2148, 0.228 to 0.0466, 0.059 to 0.0031, 0.018 to 0.0232, 0.364 to 0.7290, 0.613 to 0.4783, 0.369 to 0.2059, 0.162 to 0.0235, 0.489 to 0.2148, 0.228 to 0.0466, 0.059 to 0.0031, 0.018 to 0.0003, 0.001 to 0.2293, 0.364 to 0.7290, 0.613 to 0.4783, 0.369 to 0.2059, 0.162 to 0.0235, 0.489 to 0.2148, 0.228 to 0.0466, 0.059 to 0.0031, 0.018 to 0.0003, 0.001 to 0.0023, 0.364 to 0.7290, 0.613 to 0.4783, 0.369 to 0.2059, 0.162 to 0.0235, 0.489 to 0.2148, 0.228 to 0.0466, 0.059 to 0.0031, 0.018 to 0.0003, 0.001 to 0.0000. First rule: 0.364 to 0.8433, 0.613 to 0.7589, 0.699 to 0.6400, 0.379 to 0.4857, 0.392 to 0.3108, 0.323 to 0.1510, 0.141 to 0.0185, 0.043 to 0.0256, 0.025 to 0.0548, 0.364 to 0.8433, 0.613 to 0.7589, 0.699 to 0.6400, 0.379 to 0.4857, 0.392 to 0.3108, 0.323 to 0.1510, 0.141 to 0.0185, 0.043 to 0.0256, 0.025 to 0.0026, 0.003 to 0.1704, 0.364 to 0.8433, 0.613 to 0.7589, 0.699 to 0.6400, 0.379 to 0.4857, 0.392 to 0.3108, 0.323 to 0.1510, 0.141 to 0.0185, 0.043 to 0.0256, 0.025 to 0.0026, 0.003 to 0.0017, 0.364 to 0.8433, 0.613 to 0.7589, 0.699 to 0.6400, 0.379 to 0.4857, 0.392 to 0.3108, 0.323 to 0.1510, 0.141 to 0.0185, 0.043 to 0.0256, 0.025 to 0.0026, 0.003 to 0.0000.10⁻³10⁻²10⁻¹110⁻⁴10⁻³10⁻²10⁻¹1residual over the residual beforeforcing term chosenfirst rulesecond ruledotted: 0.9 r²damped runs, every targeta fast fall buys a tight solve
Fig. 4 The forcing term each damped Newton step was solved to against the ratio of the last two residuals, on the strong problem at every target, for the first and second rules, with 0.9 r20.9\,r^2 dotted.

The second rule’s points lie on its formula wherever neither the safeguard nor the floor binds, which on this problem is almost everywhere below a ratio of 0.3. The first rule’s points do not lie on any curve of the ratio at all. Its forcing term is ∣ ∥Fk∥−∥Fk−1+Jk−1sk−1∥ ∣/∥Fk−1∥\big|\,\|F_k\| - \|F_{k-1} + J_{k-1}s_{k-1}\|\,\big| / \|F_{k-1}\|, the amount by which the last linear model was wrong, and that is large exactly when the model has stopped being trustworthy, whatever the residual did. Where the residual fell by a factor of 23 to 40 — ratios of 0.025 to 0.043 — the first rule chose forcing terms from 0.0026 to 0.055; the second rule’s formula gives 0.0006 to 0.0017 at those ratios.

The accuracy that is thrown away opened this subject with the observation that most of an exact Newton step’s accuracy is discarded by the next step. The two rules are two estimates of how much will be discarded. The first reads it off the model’s last error; the second guesses it from the convergence rate, and the guess is quadratic in the rate. On the four milder problems the two estimates agree within a few per cent, because the convergence is steady and the rate is a good proxy for the model’s error. On the strong one it is not.

The threshold’s cost was on the other rule

As a control, the first rule’s safeguard — ηk−1ϕ\eta_{k-1}^{\phi} with ϕ\phi the golden ratio, also skipped below 0.1 — was run always applied as well. It was not part of the prediction, and it turns out to be the larger effect in this essay.

Inner iterations summed over every certifiable target, as ratios, on five problems: the second forcing rule with its safeguard always on over the default, second over first plain and damped, and the first rule with its safeguard always on over the firstκ 10⁵, λ 3, second, guard always, over default: 0.952; κ 10⁵, λ 3, second over first, plain: 1.123; κ 10⁵, λ 3, second over first, both damped: 1.123; κ 10⁵, λ 3, first, guard always, over first: 0.916; κ 10³, λ 3, second, guard always, over default: 1.000; κ 10³, λ 3, second over first, plain: 0.957; κ 10³, λ 3, second over first, both damped: 0.957; κ 10³, λ 3, first, guard always, over first: 0.885; κ 10⁷, λ 3, second, guard always, over default: 1.007; κ 10⁷, λ 3, second over first, plain: 1.012; κ 10⁷, λ 3, second over first, both damped: 1.012; κ 10⁷, λ 3, first, guard always, over first: 0.801; κ 10⁵, λ 0.3, second, guard always, over default: 1.007; κ 10⁵, λ 0.3, second over first, plain: 1.005; κ 10⁵, λ 0.3, second over first, both damped: 1.005; κ 10⁵, λ 0.3, first, guard always, over first: 0.747; κ 10⁵, λ 30, second, guard always, over default: 0.960; κ 10⁵, λ 30, second over first, plain: 1.462; κ 10⁵, λ 30, second over first, both damped: 1.356; κ 10⁵, λ 30, first, guard always, over first: 1.019.0.70.80.91.01.11.21.31.41.5κ 10⁵, λ 3second, guard always, over defaultsecond over first, plainsecond over first, both dampedfirst, guard always, over firstκ 10³, λ 3second, guard always, over defaultsecond over first, plainsecond over first, both dampedfirst, guard always, over firstκ 10⁷, λ 3second, guard always, over defaultsecond over first, plainsecond over first, both dampedfirst, guard always, over firstκ 10⁵, λ 0.3second, guard always, over defaultsecond over first, plainsecond over first, both dampedfirst, guard always, over firstκ 10⁵, λ 30second, guard always, over defaultsecond over first, plainsecond over first, both dampedfirst, guard always, over firstinner iterations, ratiofive problems, every certifiable targetthe threshold costs the first rule
Fig. 5 Inner iterations summed over every certifiable target, as ratios, on five problems: the second rule with its safeguard always applied over its default; the second rule over the first, plain and damped; and the first rule with its safeguard always applied over its default.

Always applied, the first rule’s safeguard saves 8.4 per cent on the standing problem, 11.5 on the easy one, 19.9 on the ill-conditioned one and 25.3 on the mild one, and costs 1.9 per cent on the strong one. The safeguard keeps the forcing term from falling faster than the golden-ratio power of the last one, and the threshold switches that off near the end of the run — exactly where the first rule’s own formula is noisiest, because the residuals it differences are small. How often the skipped safeguard would have bound is about the same for both rules — five to eight steps per problem over the four targets — so the difference is in how far below its safeguard each rule’s formula falls on those steps, which these counts do not measure. Neither prediction nor the essays before it priced this, and it is the one repair measured here that moves a rule’s cost by more than a few per cent on the problems where the rules already agree.

The threshold exists for a reason, and the reason can be checked. Eisenstat and Walker skip the safeguard near the end so that it cannot hold the forcing term up when the iteration has entered its fast phase and the extra accuracy is cheap and useful; a safeguard always on should cost Newton steps there. It does, but little: across the nineteen certifiable cells the always-on first rule takes one Newton step more on five and the same number on the rest, and the inner iterations those extra steps cost are more than repaid by the looser solves before them. On these problems the threshold protects a step count that it moves by one at most, and pays for that with a fifth of the inner work.

A guess worth two per cent found how little an initial guess buys a sequence of inner solves; a floor with a cliff at one found a threshold whose position decided the cost abruptly. The first rule’s threshold at 0.1 is the second kind, and it costs a fifth of the run on the ill-conditioned problem.

What the prediction weighed wrongly

The prediction located the excess in the one place the earlier measurement had pointed to, the over-tight step, and proposed to loosen that step with the rule’s own device. Three things were wrong with the weighting. The step was loosened only to the safeguard’s own value, 0.038, which was always going to be close to 0.024 on a logarithmic scale of solve accuracy. The rise that followed it was not caused by the tightness alone but by Newton’s step overshooting at that point, which any solve tight enough to approximate the Newton step reproduces. And the rise and its reset were never most of the cost: with both removed by a line search, the second rule still spends a third more, on a second over-tight step at the eighth iteration that the earlier essays had not looked at, because there was no reset after it to draw attention.

One line that buys a quarter of the run is the closest earlier result in spirit: a one-line change to the floor that saved a quarter. The change here that saves a fifth or a quarter is also one line, on the other rule.

What five problems do not show

The five problems are one family with two dials, and on four of them the residual never rises. A problem with a fold, where the Jacobian becomes nearly singular part-way through and Newton’s step is poor for several steps in a row, would test the damped comparison much harder; the step before the reset predicted that the second rule’s over-solved steps would multiply there, and nothing here contradicts that. The backtracking search is the plainest possible, a halving with no sufficient-decrease constant, and it never needed more than one halving; a problem that needed several would make the search’s own cost part of the comparison. And the scan at the step before the rise is one point on one problem. The monotone growth of a conjugate-gradient step’s length is a theorem, not a measurement, so the short-step reading of a loose solve holds wherever conjugate gradients is the inner solver; for GMRES, whose iterates’ lengths need not grow monotonically, a loose solve is not guaranteed to be a short step, and the second rule’s overshoot could be cheaper or dearer there. That the residual’s response to the forcing term is a valley with its floor near 0.35 is measured there and nowhere else.

Still open: a rule that reads the model, and the fold

The second rule fed the model’s error. If the second rule’s fault is that it reads the residual ratio rather than the model’s error, replacing rr in 0.9 r20.9\,r^2 by the first rule’s measured model error — the ratio of the model’s miss to the last residual — keeps its quadratic shape near convergence and gives it the first rule’s caution in the middle. The prediction with a sign is that on the strong problem this hybrid costs within five per cent of the first rule, needs no line search and no reset, and on the four milder problems costs within two per cent of the cheaper of the two rules.

The first rule’s safeguard on a fold. Always applying the first rule’s safeguard saved a fifth on the ill-conditioned problem by refusing to let the forcing term fall fast near the end. On a problem with a fold the forcing term should fall fast after the fold is passed, and the prediction is that there the always-on safeguard costs more than five per cent — the first problem on which the threshold at 0.1 earns its place.

Where this goes next

inner iterations in allfirst rule462second rule690second, capped675123456789101112130100200300400500600700Newton stepinner iterations so farfirst rulesecond rulesecond, cappeddotted: the step before the residual rosethe gap opens before the reset The step before the reset On the strongly nonlinear problem, inexact Newton's second forcing rule took two more Newton steps than the first and half as much work again, and the blame went to its safeguard, which let the forcing term jump back to 0.9 when the residual rose. Capping that jump at ten times the last forcing term was predicted to remove the two steps and bring the rule within five per cent of the first. It removes the steps, at three targets of four. It saves 0.7 per cent of the work, and leaves the rule 45 per cent dearer than the first. The reset was the cheap part: four loose steps for 55 inner iterations. The cost was the step before it, solved to a forcing term of 0.024 for 109 iterations, where the first rule spent 5 — and on the ill-conditioned problem, where the residual never rose at all, the cap costs ten per cent. tolerance ÷ residual reachedfirst choice, model error, unfloored median34second choice, residual ratio, unfloored median95110¹10²10³10⁴tolerance ÷ residual reachedstandingeasyhardmildstrongfirst choicesecond choicegrey: with the floordashed: stopped exactly at the testboth rules over-solve the last step Two forcing terms, one floor Inexact Newton's adaptive forcing rule comes in two versions, and every measurement of the floor under it used the second, which reads the ratio of the last two residuals. The first reads how wrong the linear model was at the last step, and it was natural to hope that a rule watching the model's own error would not solve its last step past what the stopping test needs. It does. Run without the floor on five problems at four forward targets, it over-solves its last step by a median of 34 times, against the second rule's 95; the floor saves 17.5 per cent of its inner work, against 16.0. Floored, the first rule is the cheaper by 4.6 per cent overall, and it buys that with Newton steps: two more on the median cell, a third less inner work on the strongly nonlinear problem, and 4.5 per cent more on the easy one. 33.310⁻⁴10⁻³10⁻²10⁻¹1log₁₀ numbers storeddistance to the dominant eigenvalueArnoldi, linearisedprojected quadraticper number heldstorage, linearised2080storage, second-order1040Ritz values, linearised26Ritz values, second-order52half the storageand twice the approximations A Krylov space for a problem that is not linear A quadratic eigenvalue problem has no matrix to build a Krylov space out of. The recurrence that builds one anyway stores half as many numbers, returns twice as many Ritz values — and stops being a basis at twenty vectors while the answer it gives keeps improving.

Shares its objects with

Essays that name at least two of the same things, and that neither author linked.

Named objects

A flat tag is an object no other essay names yet.

Condition numberConjugate gradientsForward errorInexact newtonStopping criterion