Numerical Linear Algebra
where the algebra stops being the arithmetic
Series

Subnormals — the series

6 essays on one idea, from the one that introduces it to the one that assumes the rest.
  1. 10⁻⁷10⁻⁵10⁻³10⁻¹⁰10⁻⁸10⁻⁶magnitudespacing to the next numberthe smallest normalgradualflush to zerosmallest normal6.1·10⁻⁵smallest subnormal6·10⁻⁸octaves of subnormals10pairs that lie under FTZ10the spacing stops halving and stays putwhich is what makes x − y = 0 mean x = y

    The numbers below the smallest one

    Below the smallest normal number the spacing stops halving and stays put, all the way to zero. That is what gradual underflow is, and the thing it buys is the sentence every algorithm assumes without being told — x minus y is zero only when x equals y.

    part 1 · arithmetic
  2. deepest barrier, far well right to within halfno exponent limit40gradual, started at 102433fp16, gradual underflow23fp16, flush to zero13051015202530354010⁻⁵10⁻⁴10⁻³10⁻²10⁻¹110¹barrier depth, octaves below the near wellrelative error in the far well's masssmallest normalsmallest subnormalno exponent limitgradual, started at 1024fp16, gradual underflowfp16, flush to zeroan error of one: the far well is emptysubnormals bought nine octaves

    The well on the far side of the band

    Gradual underflow was said to buy a predicate and not an answer, because a quantity that has decayed into the subnormal range is already lost. A quantity that passes through the band on its way somewhere else is not. The stationary distribution of a two-well chain, computed in half precision across a barrier whose top is two to the minus sixteen of the near well, keeps its far well's probability of 0.2454 to three digits with subnormals and returns exactly zero without them. The normwise backward error calls both answers exact.

    part 2 · arithmetic
  3. 0510152025303540first barrier depth at which the far well is wrong by half, octavesfp16the band adds 11 octaves · starting at the top adds 15E4M3the band adds 2 octaves · starting at the top adds 10E5M2the band adds 3 octaves · starting at the top adds 14flush to zero13gradual underflow24flush, started at the top29gradual, started at the top39flush to zero5gradual underflow7flush, started at the top13gradual, started at the top17flush to zero13gradual underflow16flush, started at the top27gradual, started at the top30dot: median of nine chains · bar: their rangethe headroom is worth more than the band

    The far well in a byte

    Half precision's subnormals carried a two-well chain's far well ten octaves deeper than flush to zero, and the essay that measured it predicted that on the eight-bit formats the band would be worth almost nothing and starting high almost everything. Measured on nine chains, the band is worth what it always was, two or three octaves; starting at the top of the range is worth ten and fourteen; and at three significant bits the far well is lost on eight chains of nine with no exponent limit at all, because the precision fails before the range does.

    part 3 · arithmetic
  4. 0102030405060first barrier depth at which the far well is wrong by half, octavesfp16E4M3E5M2gradual underflow, from onegradual, from the topcounter, floor and ceiling9 of 9counter, window from the rates9 of 9gradual underflow, from onegradual, from the topcounter, floor and ceiling1 of 9counter, window from the rates8 of 9gradual underflow, from onegradual, from the topcounter, floor and ceiling1 of 9counter, window from the rates1 of 9ring: chains that keep the far well past sixty octavesthe window is the unbounded exponent

    Room for the step, not the value

    Starting a recurrence at the top of an eight-bit format's range spends its headroom once. An exponent counter beside the stored value was predicted to spend it at every step, remove every range failure and leave only the precision. Rescaled on the value — back to one whenever it nears the bottom or the top — it does not: E4M3 still loses its far well on seven or more chains of nine, because one climbing step multiplies the value by forty and E4M3 clips silently at 448. Rescaled into a window read off the chain's own rates, it reproduces the unbounded exponent bit for bit on all 531 runs of every format, and what is left of E4M3's limit is nothing.

    part 4 · arithmetic
  5. 020406080100120barrier depth, octaves32 statesa rate is zero from 9140 statesa rate is zero from 11548 states56 states64 stateswindow under one octavewindow width below zerothe window rule first failsthe aiming rule first failsthe window per step: exacta stored rate is zeroE4M3, five chain lengths, every depth from 2 to 120a window needs an octave to hold a power of two

    An octave to land in

    A window read off a chain's own rates kept an eight-bit recurrence exact to sixty octaves, and its width, falling towards zero, was predicted to mark where it would stop being exact, to within an octave. It stops two to seven octaves earlier, as soon as the window is narrower than the octave a power-of-two rescale moves by. The rule that aims every step, predicted to outlast it, fails too, once the value it aims at no longer fits. A window computed for each step separately never narrows below 4.8 octaves, reproduces the unbounded exponent on every depth at which the format can still store the chain, and makes the fewest rescales of the three.

    part 5 · arithmetic
  6. blocks of 232 states×40 states×48 states×56 states×64 states×blocks of 432 states×40 states×48 states×56 states×64 states×blocks of 832 states×40 states×48 states×56 states×64 states×020406080100120barrier depth, octaves×the predicted lineblock window under an octaveblock window emptyfirst stops being exactexacta stored rate is zeroE4M3, every depth from 2 to 120the block fails where its window empties

    Twice the block, half the depth

    An eight-bit chain kept exact by a window per step was predicted to keep exactness with one exponent shared by two states, and to lose it with four from the depth where four of its steepest steps span more than the narrowest window, 4.8 octaves. Blocks of two fail on four chains of five, at 66 to 114 octaves. Blocks of four fail at 35 to 66, two to three times deeper than the predicted 13 to 26. Every block size fails within three octaves of the depth where its block's window empties, and that depth is 4.0 to 4.6 octaves for each state a block holds.

    part 6 · arithmetic

All series