Numerical Linear Algebra
where the algebra stops being the arithmetic
Concept

Expression contraction — where it appears

A compiler's substitution of a fused multiply-add for a separate multiply and add, permitted by C and Fortran and recorded nowhere in the source. Both forms conform to IEEE 754 and differ by one rounding of the product, and which one a binary contains is not visible from the program.

Named by 3 essays across one field — each of them below, with the objects they name alongside it.

Also named here as fused multiply-add — the same set of essays touches all of them, so they are one junction rather than several.

2022242628303210⁻¹⁷10⁻¹⁴10⁻¹¹10⁻⁸10⁻⁵10⁻²k, where the entries are near 2ᵏrelative error of the determinantas writtenfused: exactproducts need 54 bitsone rounding, the whole answertrue determinant1naive, k = 300fused, k = 301first wrong at k27sizes returning 06both forms conformand the source does not say which

One multiply the compiler removed

A determinant whose value is exactly 1, computed as exactly 0 by the expression that is written down, and exactly 1 by the same expression with the multiply and the add fused. Both forms conform to IEEE-754, both are legal compilations of the same source, and nothing in the program says which one you have.

machine · fma contraction -12-11-10-9-8024681012log₁₀ of how nearly dependent the columns areverdicts that disagreed, of 40724103which one is correctmatrices tested200verdicts disagreed26fused was right9unfused was right17lower part: thefused build was righta sign has no last digitso a verdict has nowhere to hide

A matrix that is definite on one machine

Two hundred Gram matrices, two conforming builds, and twenty-six of them get different answers to "is this positive definite". The exact verdict, from determinants in BigInt rationals, says the fused build is right nine times and the other one seventeen.

machine · fma contraction -101-101(x − 1) ÷ 0.001p(x) ÷ 2.68·10⁻¹⁵a square, drawn negativepoints sampled401negative, unfused179negative, fused196signs disagree at98widest gap1.4·10⁻¹⁵unfusedfusedthe true valuethe true curve is the flat oneeverything else is rounding

A square that evaluates negative

(x − 1)⁶ evaluated near x = 1 comes out negative at 179 of 401 points on one build and 196 on another, and the two disagree about the sign at 98 of them. Neither is nearer the truth: both traces are made entirely of rounding.

machine · fma contraction

Named alongside it

The objects these essays reach for when they reach for this one.

Fused multiply-addBitwise reproducibilityCatastrophic cancellationDefiniteness testBisectionCholesky factorisationCondition squaringDeterminantError accumulationError-free transformationExact arithmeticGram matrix

All concepts