Numerical Linear Algebra
where the algebra stops being the arithmetic
Series

Balanced truncation — the series

6 essays on one idea, from the one that introduces it to the one that assumes the rest.
  1. 123456710⁻¹¹10⁻⁹10⁻⁷10⁻⁵10⁻³10⁻¹order r kepterror and bounds2Σσ, and the errorσᵣ₊₁a bound that is an equalityorders7bound ÷ error, worst1spread over the sweep1error ÷ σᵣ₊₁, worst2.1computed before the modeland attained by it

    The bound that is known in advance

    Almost every error in numerical linear algebra is measured after the fact. Balanced truncation has one that is computable before the reduced model exists, in a norm of a function rather than of a residual — and on ordinary problems it is not an upper bound that is loose. It is attained.

    part 1 · reduction
  2. 1357911131510⁻¹⁸10⁻¹⁵10⁻¹²10⁻⁹10⁻⁶10⁻³1index kσₖ ÷ σ₁σ₁√utwo factorstheir productthe normal equations, againroutes agree to k =6product floors at9.7·10⁻¹⁰σ₁√u2.3·10⁻⁹κ(P)κ(Q)1.8·10³⁶√ of it1.3·10¹⁸do not form the productthe σ below the line are the bound

    The product nobody had to form

    The Hankel singular values are the square roots of the eigenvalues of PQ. Form that product and half of them stop existing, at a floor one number predicts — and the fix is the one the least-squares field has had since its first essay, arriving in a place with no least-squares problem in it.

    part 2 · reduction
  3. ‖H − Hₙ₋₁‖∞ ÷ σₙpositive residues2.0000mixed residues2.0000clustered poles2.0000four poles2.0000wide spectrum2.0000‖ΔH‖ 3.89·10⁻⁴ σ 1.94·10⁻⁴‖ΔH‖ 3.92·10⁻⁴ σ 1.96·10⁻⁴‖ΔH‖ 0.00502 σ 0.00251‖ΔH‖ 8.41·10⁻⁴ σ 4.21·10⁻⁴‖ΔH‖ 4.6·10⁻⁴ σ 2.3·10⁻⁴2four systems, four ratios, one number: 2.0000one state, one equalityand the sum is that, repeated

    The state that is removed is not a mode

    Balanced truncation removes one state and pays exactly twice one Hankel singular value. The natural reading is that the state removed is the model's least important mode and that the σ is that mode's own size — and on five systems that reading over-estimates by between 1.21 and 7.46, never once under.

    part 3 · reduction
  4. ζ = 0.03largest over-statement4.8removing one state1123456789101112345order keptbound ÷ measured errordashed: the bound attaineda mode that rings is not a heat mode

    A mode that rings is counted twice

    Balanced truncation's bound, twice the sum of the discarded Hankel singular values, was attained at every order on the heat model — a formula for the error rather than a bound on it. On six lightly damped oscillators it is loose at every order but the last, by factors between 1.3 and 5, at every damping from 0.7 to 0.01. The reason is visible in the singular values: a mode that rings contributes two of them, nearly equal, and removing the mode costs about what removing one would. The last mode's bound over its error is 2.02, 2.00 as the damping falls to 0.03 and 0.01. The equality was a property of modes with one singular value each.

    part 4 · reduction
  5. the pair's two σ added: distance from oneworst, damping to 0.050.053worst at a resonance, damping 0.10.160.40.60.811.21.41.61.8damping ratioreading ÷ measured error0.0050.010.030.10.3order 2order 4order 6order 8order 10shaded: within ten per centa pair costs its two σ, until the modes overlap

    Where a ringing mode stops being a pair

    A lightly damped mode puts two nearly equal Hankel singular values into balanced truncation, and the reduction error was read as twice the larger of the first pair cut — right to two per cent at damping 0.01, wrong by half at 0.1. Read instead as the pair's two values added, the error is right to within six per cent at every order up to damping 0.05, and the pair's split turns out to be twice the damping. What it cannot do is say when the reading fails. The split predicts nothing: the widest split at damping 0.1 has the best reading. The failures come from somewhere the pair does not look — the other dropped modes' response under the peak, and a cut between two singular values that agree to under one per cent, which left a reduced model with a pole a thousand times closer to the axis than any pole of the system it came from.

    part 5 · reduction
  6. the censusreductions1452ties431doubled, all ties1010⁻³10⁻²10⁻¹110¹10²0.511.52gap at the cut ÷ smaller neighbouring gaperror ÷ twice the first dropped σdoubled abovetiesshaded: gap under a fifth of its neighboursevery doubled error is at a tie

    A tie is a gap smaller than its neighbours

    A balanced truncation that cut between two Hankel singular values agreeing to under one per cent left a reduced model with a pole near the axis and twice the error the σ promised, and the warning drawn from it was a rule: never cut inside a tie. Over 1,452 reductions of twelve oscillators, all ten doubled errors are at ties — but three of them have gaps of about two per cent, which a one-per-cent rule lets through, and what they share is a gap under a fifth of the gaps beside it. Moving every such cut one order down never doubles and halves each doubled error, for a median cost of one per cent. It is not free everywhere: at damping 0.5 it costs up to two and a half times. And the near-integrator was never the harm: 32 reductions have one and 22 of them read their σ correctly. The cheapest test of all is one evaluation of the error at zero frequency, which picks out the ten doubled cuts and nothing else.

    part 6 · reduction

All series