Numerical Linear Algebra
where the algebra stops being the arithmetic
Concept

Matrix polynomial — where it appears

A polynomial whose coefficients are matrices, so that its value at a number is a matrix. The spectrum belongs to the polynomial rather than to any one of its values, and there is nothing to factorise because the object is not a matrix.

Named by 12 essays across 3 fields — each of them below, with the objects they name alongside it.

00.0939569-2-1.24498-0.4899610.2650581.020081.7751real partimaginary parttwo routes, one spectrumeigenvalues16rows8complex16against the closed form2.1·10⁻¹⁵n rows and 2n eigenvaluesso the eigenvectors are not a basis

A matrix that depends on its own eigenvalue

A damped structure does not produce Ax = λx. It produces (λ²M + λC + K)x = 0, where the matrix whose null vector is wanted is a function of the number being solved for — so there is nothing to factorise, an n × n problem has 2n answers, and the eigenvectors cannot be a basis.

polynomial · Polynomial eigenvalue 0246810⁻¹⁷10⁻¹⁴10⁻¹¹10⁻⁸10⁻⁵10⁻²10¹log₁₀ γ, the change of unitsrelative errorforward errorη, the quadraticη, the linearisationagainst a closed formη(linearisation), worst7.6·10⁻¹³η(quadratic), worst1.2·10⁻⁴forward error, worst0.0013coefficient spread4.2·10¹⁵the solver is right at every stopabout a problem nobody asked

A backward-stable answer to a problem nobody asked

One quadratic eigenvalue problem, in nine systems of units, with a change of variable that is exact in both directions. The residual the solver prints stays at the rounding level at every stop. The answer loses eleven orders of magnitude, and the two facts are consistent.

polynomial · Linearisation backward error 0246810⁻¹⁶10⁻¹³10⁻¹⁰10⁻⁷10⁻⁴10⁻¹log₁₀ γ, the change of unitsforward erroras writtenafter scalingone change of variableunscaled, worst0.0013scaled, worst1.7·10⁻¹³orders recovered10scaled coefficient spread4.5the answer was never the problemthe units were

The scaling that buys ten orders

Two lines computed from three norms, a change of variable that is exact in both directions, and the whole of the loss the previous essay measured comes back — flat, at every stop, because after scaling every stop is the same problem.

polynomial · Polynomial scaling first · leading5.47·10⁻⁶first · trailing4.62·10⁻⁵second · leading4.27·10⁻⁵second · trailing2.23·10⁻⁴symmetric · leading5.47·10⁻⁶symmetric · trailing4.05·10⁻⁵all six are the same algebrabest route5.5·10⁻⁶worst route2.2·10⁻⁴spread across the six41condition of the linearisation8.3·10¹²the spectra agreeand the arithmetic does not

Six routes to one spectrum

Three linearisations of one quadratic, each reduced to a standard eigenvalue problem two ways. All six have exactly the same eigenvalues in exact arithmetic. On a well-scaled problem they differ by noise; on a badly scaled one by a factor of forty; and two of the six are the same matrix.

polynomial · Linearisation backward error -2-101234-5-3-1135real partimaginary part4 insidea countable spectruminside the contour4drawn12existing∞worst branch residual1.6·10⁻¹⁵there is no last eigenvalueso the question has to change

A problem with infinitely many eigenvalues

Let the matrix depend on λ through something that is not a polynomial and three things stop being true at once. There is no linearisation, there is no characteristic polynomial, and "compute the spectrum" is not a request that can be granted — the only finite question is how many eigenvalues are inside this circle.

polynomial · Nonlinear eigenvalue 57911131517192110⁻¹⁷10⁻¹⁴10⁻¹¹10⁻⁸10⁻⁵10⁻²degreerelative error of the worst rootmeasuredκ × ua polynomial is its coefficientsworst root, measured0.0076predicted, κ × u0.0023root condition number10¹³largest coefficient1.4·10¹⁹the roots are integersand the coefficients are not the roots

The roots are not the coefficients

A polynomial whose roots are the integers one to twenty, expanded exactly, handed to the routine every library uses. The computed roots are wrong in the third digit, the computation is backward stable for the matrix it factorised, and above degree eighteen the coefficients are not double-precision numbers at all.

error · Conditioning 11.31.610⁻¹110¹10²10³log₁₀ ntropical root ÷ actual modulusexactsmallest moduluslargest modulusthree norms, two endslarge root, n = 40.96large root, n = 320.94small root, n = 45.4small root, n = 32230a maximum predicts a maximumand says nothing about a minimum

An estimate that does not move

The tropical roots are said to miss the bottom of a spectrum by a factor growing like n². The estimate does not get worse. It changes by four per cent between four masses and sixty-four while the modulus it names falls by a factor of a hundred and sixty, and an estimate that is constant in the variable the answer depends on is a different defect from an inaccurate one.

polynomial · Polynomial scaling finite eigenvalues (degree of det Q)13at infinity (2n − degree)1at infinity, by the rank of M12n, if M were nonsingular14a degree, not a decisiondegree of det Q13at infinity1by the rank of M1singular-value gap∞the count is a degreeand the other route is a judgement

One mass removed, and one eigenvalue gone

A coordinate with no inertia reads like a coordinate that has been deleted, and a chain of eight masses with one of them removed would then be a chain of seven, with fourteen eigenvalues. It has fifteen. The massless coordinate is still there, still carrying a damper, and it contributes a first-order equation rather than none.

polynomial · Polynomial eigenvalue 02468101201change of units, by exponent10⁰10¹10²10³10⁴10⁶10¹⁰10¹⁶10¹⁹10²⁰10⁴⁰10¹⁵⁰as writtenafter scalinga range questionlargest finite value3.4·10³⁸predicted boundary γ1.8·10¹⁹last γ that forms10¹⁹stops where scaling fails0not a poor answerno answer at all

The units that overflow before the answer does

A change of variable that is exact in the algebra requires γ² times a matrix to be a number the format can hold. In binary64 that is a bound nobody meets by accident. In binary32 it arrives at 10¹⁹ and in fp16 at 256, and past it there is no answer rather than a poor one.

arithmetic · Overflow 012345610⁻¹10²10⁵10⁸10¹¹log₁₀ γ, the change of unitscondition numberthe linearisationthe quadraticone problem, two amplifiersκ(quadratic), first6.6κ(quadratic), last6.6κ(linearisation), last10¹²how far the first moved1the problem is as well conditioned as everand the method is not

The number that moves when the problem does

Two quantities are offered as the condition number of one eigenvalue. One is unmoved to eight digits by a change of variable that is exact in both directions, and grows like the square root of the chain's length. The other is inflated by ten orders by that change of variable, and is ten times too large before anything has been done at all.

polynomial · Linearisation backward error small group, worst backward errorleading, at 4·10¹¹2.7·10⁻⁷flv, at 4·10¹¹7.1·10⁻⁷tropical, at 4·10¹¹10⁻⁵trailing, at 4·10¹¹7.1·10⁻¹⁶both, at 4·10¹¹7.1·10⁻¹⁶10¹10⁵10⁹10¹³10¹⁷10²¹10⁻¹⁷10⁻¹⁴10⁻¹¹10⁻⁸10⁻⁵10⁻²separation ‖C‖²/(‖M‖‖K‖)backward errorleading reductionleading, FLV scalingleading, tropical scalingstrailing reductionboth reductionsdotted grey: u times the separationonly the trailing reduction keeps the small group

Two groups need two reductions

When a quadratic eigenproblem's tropical roots are far apart its eigenvalues fall into two groups, and the remedy offered is a second scaling — one per tropical root, each run keeping the group its root predicts. Raise the damping on an eight-mass chain until the roots are 10²³ apart and measure every eigenvalue: no scaling rescues the small group. Its backward error grows as the unit roundoff times the separation under Fan–Lin–Van Dooren's scaling, under both tropical scalings and under none, to 10⁻⁵ at a separation of 4·10¹¹. What rescues it is the other reduction of the same pencil — inverting the constant term instead of the leading one — which keeps the small group at rounding at every separation and loses the large one instead. Both reductions, each keeping its own group, give all sixteen eigenvalues to 10⁻¹¹ at worst. On the way, the closed form that served as the exact answer turned out to lose the small roots to cancellation, from the same separation.

polynomial · Polynomial scaling 1e-161e-121e-81e-41e0worst backward error over the groupleading reductiontrailing reductionscaled at the middle root, leadingscaled at the small root, leadingsmall groupmiddle grouplarge groupsmall groupmiddle grouplarge groupsmall groupmiddle grouplarge groupsmall groupmiddle grouplarge groupdots: worst over five cubicsthe middle group comes with the large one

The middle group needs no reduction of its own

A quadratic whose two groups of eigenvalues sit far apart needs two reductions, one per end. A cubic can have three groups, and the middle one is inverted by neither reduction, so the worry was that it would need a third route — a shift-and-invert at the middle tropical root. Build order-four cubics with three groups up to eight decades apart and measure every eigenvalue: the leading reduction keeps the middle group at 10⁻¹¹ or better at every separation, the trailing reduction loses it to 5·10⁻⁹ by eight decades, and the reversed polynomial's leading reduction, which inverts the same constant coefficient as the trailing one, keeps it at 2·10⁻¹⁴. The middle group's loss belongs to the pencil, not to the end. Two reductions still give all twelve eigenvalues to 2·10⁻¹¹. And the tropical scaling at the small root, which rescues the small group when every coefficient shares one diagonal form, does nothing for it once the coefficients are perturbed out of that form.

polynomial · Polynomial scaling

Named alongside it

The objects these essays reach for when they reach for this one.

Quadratic eigenvalue problemExact ground truthScalingLinearisationBackward errorCompanion formCondition numberTropical rootsAsymptotic analysisFrobenius normA-priori boundCharacteristic polynomial

All concepts