The far well in a byte

The well on the far side of the band found that a stationary distribution can cross the subnormal band and come back out of it. A two-well birth–death chain computed in half precision keeps its far well for ten octaves of barrier longer with gradual underflow than with flush to zero, because the far well’s probability is built from the small values at the bottom of the barrier and gradual underflow keeps those small values a few bits short rather than setting them to zero. It measured three remedies — the band, a recurrence started high in the range, and an exponent with no limit — and found that each is worth what the format’s own geometry says it should be: the band its width of $p - 1$ octaves, the start the octaves of headroom it uses, and the unbounded exponent everything.

It closed on a prediction about the two eight-bit formats now used for exactly the probability-like quantities a recurrence of this kind computes. E4M3 has four significant bits and a band of three octaves; E5M2 has three and a band of two. “The law above predicts that on those formats the subnormals are worth almost nothing and starting high is worth almost everything.” It also put numbers on the first of them: the same chain would lose its far well in E4M3 “past a barrier of six octaves without subnormals and past about nine with them.”

The prediction treats the byte as half precision with less of everything. That is half of what happens. The other half is that half precision had one property the byte does not — eleven bits is enough precision that the precision never shows — and at three bits the order in which the three ingredients run out is different.

The same chain, stored in a byte #

The chain is the one the earlier essay built: a double-well potential, $\sin^2(\pi s)$ plus a slight tilt, with Metropolis rates and a temperature set so that the top of the barrier has probability $2^{-d}$ of the near well’s bottom. The stationary vector comes from detailed balance by forward substitution, $\pi_{i+1} = \pi_i\,p_i/q_{i+1}$, and the quantity asked for is the share of the probability in the right half of the chain — 0.34 at eight octaves, falling slowly as the barrier deepens.

Three things change at eight bits. The rates are stored in the format being measured, and the reference is the same recurrence in double precision on those stored rates, so as before every difference is the arithmetic’s and none is the data’s. The two sums that turn a vector into a share are accumulated in binary32, because that is what eight-bit hardware does with an eight-bit product, and a later section measures what happens if they are not. And one chain is no longer a measurement. A rate rounded to three significant bits can move by a sixth of itself, which moves the barrier, and the far well’s error then jumps from one depth to the next by more than any trend. Every survival depth below is read on nine chains, from 32 states to 64 in steps of four, as the median depth at which the far well is first wrong by half, with the range across the nine.

The five routes are the earlier essay’s, with one added. Gradual underflow and flush to zero, each started at one; each started instead at the largest power of two the format holds, which is 256 in E4M3 and 32,768 in E5M2 and half precision; and the format’s significand with no limit on its exponent at all.

The band is still worth its width #

The figure at the top of the page is the whole of the comparison. In half precision, on nine chains rather than one, gradual underflow carries the far well to a median of 24 octaves and flush to zero to 13: the band is worth eleven, against a width of ten, which is the earlier essay’s law with its octave of slack from the rounded rates.

The eight-bit rows are the same law. In E4M3 flush to zero loses the far well at a median of five octaves and gradual underflow at seven: the band is worth two octaves against a width of three. In E5M2 the two are thirteen and sixteen: three against two. Both are within an octave of $p - 1$, which is exactly as close as the half-precision measurement came. The band was never worth a fixed share of the format; it was worth $p - 1$ octaves of barrier, and a format with fewer bits has a narrower band and gets fewer octaves from it in proportion.

So the first half of the prediction is wrong in the way that matters. “Almost nothing” read the band as worthless because it is small beside the format’s whole range. Against the barrier it is worth two or three octaves, and for a quantity like this one that is a factor of four to eight in how unlikely a transition the chain can carry before its far well is gone.

The dial shows what two octaves of band are buying. At four octaves nothing reaches the band and the three computations agree to the format’s few bits. At six, flush to zero zeroes the state at the bottom of the barrier and every state after it, and gradual underflow carries the chain through. At eight the bottom of the barrier is in the band’s last octave: gradual underflow still gets the far well across, but at 0.17 against an exact 0.34, and the flat run of dots along the bottom of the barrier is the mechanism the earlier essay called sticking — a value that should keep falling rounds back up to a few units of the smallest subnormal and sits there — here at $2^{-7}$ for nine states and one step higher for eight more, while the exact values fall to less than half of that and climb back past it. It errs downwards this time rather than upwards, because the run ends below where the exact climb has got to, and the climb out multiplies the shortfall.

Why six octaves became five #

The earlier essay’s numbers for E4M3 were six octaves without subnormals and nine with them, from the law: flush to zero fails when the barrier’s top falls below the smallest normal, $2^{-6}$, and the band adds three. The nine-chain medians are five and seven, and both shortfalls have a cause worth having.

The smallest number the recurrence ever holds is not the dip. It is the product $p_i\,\pi_i$ formed before each division, and on the climb to the top of the barrier the rate $p_i$ is close to a half, so the product sits one octave below the probability it is about to become. In half precision that octave is invisible in the median — the dip at thirteen octaves puts the product exactly on $2^{-14}$, which is representable, and flush to zero fails at fourteen. In E4M3 the rates are rounded to four bits, and at a nominal five octaves the computed dip is at $2^{-4.92}$ and the product at $2^{-5.92}$, a twelfth of an octave above the smallest normal. The rounding of the products themselves, at one part in sixteen, carries it under. The law’s first number is right; the format’s precision costs it an octave, because at four bits a twelfth of an octave is inside the rounding.

The second shortfall is the band’s. A three-octave band is three octaves of numbers with fewer and fewer significant bits: the smallest subnormal has one, and the one above it two. Half precision’s band ends in one-bit numbers too, but it has eleven bits to lose before it gets there and the far well was right to a few per cent across most of it. E4M3 starts the band with four bits and the far well already a few per cent off, so it is wrong by half one octave earlier. That is a precision effect inside the range effect, and it is the first sign of the theme of the rest of the page.

Starting high is worth more than the band, on every format #

The second half of the prediction is right, and the figure above says how right. Starting the recurrence at $2^k$ rather than at one moves every value up $k$ octaves, so the bottom of the barrier reaches the band $k$ octaves later, and in half precision the survival depth rises by exactly one octave per octave of start, from 24 at $k = 0$ to 39 at $k = 15$. E5M2 has the same exponent range as half precision and gains fourteen octaves over the same fifteen, wandering by an octave at $k = 10$ where a three-bit rate happens to land differently. E4M3 has eight octaves of headroom above one and gains ten from them.

Set beside the band, the headroom is worth about five times as much in E4M3 and E5M2 and about a third more in half precision. That ordering was already true at half precision, where the earlier essay started at 1,024 and found the two remedies roughly equal, ten octaves each. What the byte changes is the ratio, and the reason is that the two remedies scale with different things. The band is $p - 1$ octaves and shrinks with the significand. The headroom is the upper half of the exponent range and does not care how many significant bits there are. A byte keeps most of its exponent and gives up most of its significand, so it keeps most of the headroom and loses most of the band.

The arithmetic of where the headroom comes from is also what limits it. A normalised recurrence that starts at one spends none of the range above one, and in every format here that is roughly half of the exponent range: sixteen octaves of half precision’s forty, nine of E4M3’s eighteen, sixteen of E5M2’s thirty-two. The other half of a format called the exponent field a budget, and a probability vector normalised at its largest entry has decided in advance to spend only the half of it below one.

Gradual underflow and the high start compose, as they should. In E5M2 flush to zero started at the top survives to 27 octaves against gradual’s 30; in E4M3 to 13 against 17. The headroom moves the barrier at which the band is reached and the band then adds what it adds there, so the composed survival is roughly the sum of the two gains.

And at three bits, the significand loses the far well by itself #

The route the earlier essay used to show that the exponent alone was responsible was the unbounded one: the same eleven-bit significand with no limit on the exponent kept the far well to within $3.3 \cdot 10^{-3}$ at every depth to forty, while the normwise condition number passed $10^8$. It isolated the range by taking it away, and what was left was the precision, which at eleven bits was a few units of roundoff.

At eight bits that control stops being clean. The figure is 531 unbounded runs for each format — nine chain lengths at every depth from two to forty octaves — and half precision sits where it did, at a median relative error of $7.4 \cdot 10^{-4}$ and a worst of $4.5 \cdot 10^{-3}$. E4M3’s four bits put the median at 0.080: more than a third of all runs are wrong by a tenth or more, and one chain of the nine loses its far well to half at 38 octaves with no range limit at all. E5M2’s three bits put the median at 0.146 and a tenth of runs above 0.38; a quarter of all runs are wrong by a quarter or more, and eight of the nine chains are wrong by half at some depth, the first at 21 octaves and the rest scattered between 30 and 45.

The reason is the recurrence’s own length rather than anything about the barrier. Every state’s probability is the product of its predecessor’s with one rate ratio, so the far well’s states carry the rounding errors of every multiplication and division on the way to them — ninety-four operations for the last state of a 48-state chain. About half are exact, because a Metropolis rate of exactly one half multiplies and divides without error; on the 48-state chain at ten octaves, 51 of the 94 are. Each of the other forty-odd can be off by one part in eight at E5M2’s unit roundoff of $2^{-3}$. They are not all in the same direction, so they do not add, but they do not cancel either, and forty independent errors of up to an eighth wander by a few tenths. The componentwise condition number the earlier essay measured is still $2n - 1$ for every state, and the earlier essay found the problem perfectly well posed. It still is. The arithmetic is simply too coarse to deliver a well-posed answer to three significant figures, or to one.

This is the part the prediction could not see from half precision, because there the third ingredient was free. The question “which barrier does a format survive” assumes the format would survive any barrier if it had the range. At three significant bits it would not.

On a single chain the two effects lie on top of each other. The 48-state chain in E5M2 is, by luck of its rounded rates, one of the chains that never crosses one half with an unbounded exponent; its worst is 0.49, at 27 octaves. Its other routes follow the unbounded one exactly until they reach their limit, because until then they are the same arithmetic. Flush to zero and gradual underflow both first fail at fourteen octaves: the band’s two octaves, on this chain, buy nothing at a tolerance of a half, and gradual underflow recovers to 0.35 at fifteen before failing for good at sixteen. The high start follows the unbounded line all the way to thirty and fails at 31. Every one of the four routes is wandering between one per cent and half its value the whole way, which is the noise in the previous figure seen one chain at a time.

What that does to the question of how deep a barrier a format survives is worth saying exactly. In half precision the answer was a depth, the same on every chain to within two octaves. At three bits it is a depth plus a coin: a tolerance of a half is about where the precision alone starts to fail, so the survival depth is partly a property of the range and partly a property of which rates rounded up. At a tolerance of a tenth the question has no answer for E5M2 at all — 62 per cent of the unbounded runs miss it at depths where nothing is near the band.

Summing in the byte loses it before any barrier does #

Everything above accumulated the far well’s share in binary32, and that choice is the reason the figures show anything at all. Done in the format, with each partial sum rounded back to a byte, the share is wrong at the shallowest barriers measured. In E5M2 at two to five octaves of barrier — where the binary32 sum is right to between one and two per cent — the in-format sum is wrong by 1.21, 1.14, 1.14 and 0.91; in E4M3 at two to four octaves it is wrong by 0.56, 0.36 and 0.36, against at most 0.07.

The mechanism is stagnation, and it is a statement about the precision with no range in it. The near well’s probabilities start at one and fall slowly, so a running total of them passes eight within ten terms, and at three significant bits the numbers between eight and sixteen are two apart: a term below one, which is every term after the first, rounds to nothing or to a whole spacing when it is added. The true totals here are between 13 and 25. The sum stops tracking them at a value the format decides, and the share is that value’s ratio to another such sum. The vector that hides it put the general rule as a condition number — a sum’s reproducibility is decided by the total size of its terms over the size of its answer — and here the number is one and the sum still fails, because the other half of a sum’s accuracy is how many terms there are against how many bits the total has. Adding a term near one to a total near twenty, and keeping any of it, takes five or six significant bits in the total, and a byte has three.

That is why eight-bit hardware does not do it. A product of two eight-bit numbers is exact in a wider accumulator, and every matrix unit that offers E4M3 or E5M2 accumulates in binary32 or at least half precision. Where the hardware went traced the same design decision from the other end: the narrow format is for storage and multiplication, the wide one for anything that adds. A recurrence like this one is a product chain and survives in a byte to the extent the previous sections say; its normalisation is a sum and does not survive in a byte at all.

What each format is good for, for this computation #

The measurements turn into a short table of what to expect from each format on a probability recurrence that crosses a barrier, with its sums accumulated wide.

Half precision is limited by its range and nothing else. Flush to zero is good for thirteen octaves of barrier, the band adds ten or eleven, and starting at the top of the range adds fifteen more — to 39 octaves of barrier, a transition probability below $2 \cdot 10^{-12}$, with every far well right to a few units of roundoff. Two condition numbers of one matrix is the account of why the normwise condition number of the balance equations says otherwise and is wrong.

E4M3 is limited by its range first: flush to zero fails at five octaves, gradual underflow at seven, and the high start, from 256, at seventeen. Its precision costs a typical far well eight per cent and occasionally a third. It is a format for a barrier of up to about $2^{-16}$ if started high, and for a well-posed answer to about one significant figure.

E5M2 has the range of half precision and fails like it — thirteen, sixteen and thirty octaves on the three routes — but its precision makes those depths soft. The median far well is fifteen per cent wrong at every depth, and on eight chains of nine some depth that the range handles easily is wrong by half anyway. It is a format in which a recurrence of this length cannot promise one correct significant figure, at any barrier, by any treatment of the exponent.

The pairing eight bits and a format that breaks the rules described — one format for values and one for gradients, the two axes bought separately — is visible here as two different failures. E4M3 spends its bits on precision and runs out of range at a shallow barrier. E5M2 spends them on range and has none of the first left. Neither has enough of both to carry a 48-state recurrence across a barrier the way half precision can, and the remedy that helps most in both is the one that costs nothing: starting the recurrence at the top of the range instead of at one.

What the earlier reading got right, and what it missed #

The prediction’s second half survives intact and with a larger margin than it claimed: the headroom is worth five times the band at eight bits, and starting high is the single largest remedy on every format measured. Its first half fails on a misreading of the law it came from. The band was always worth $p - 1$ octaves of barrier, and $p - 1$ at eight bits is two or three, which is small beside the headroom but not nothing beside the barrier — a factor of four to eight in the transition a chain can carry.

What it missed is not about the band or the headroom. It is that the earlier measurement’s control — the same significand with an unbounded exponent — was a perfect isolation of the range only because eleven bits made the precision invisible. At three bits the control fails on its own, and the clean decomposition into a precision part that does not matter and a range part that does stops being available. The exact answer to a nearby problem is right that a well-posed problem solved stably gets an answer as good as the precision allows. At three bits, as good as the precision allows is a fifth.

What nine chains do not show #

Nine chain lengths of one potential and one tilt. A different shape of barrier — a sharper one, or one whose climb rates sit far from a half — would move the product before each division further below the dip and shift every flush-to-zero depth by up to an octave. The sums are accumulated in binary32 throughout, which is the common hardware choice and not the only one; a half-precision accumulator would behave like binary32 at these lengths and differently at a few hundred states. The formats are simulated by rounding each operation’s exact result, which is how E4M3 and E5M2 are specified and not how every device implements them; a device that rounds a product in two steps, or saturates E4M3 rather than returning NaN past 448, would agree with these numbers wherever nothing reaches the top of the range, which in this recurrence is everywhere. Stochastic rounding, which some eight-bit units offer precisely because of stagnation, is not measured.

Still open: several barriers, a rescaled recurrence, and a cheap certificate #

A barrier crossed twice. The earlier essay’s second open question stands and the byte makes it sharper. Three wells in a row put two barriers on the path to the last one, and if each barrier’s bottom bills its relative error to every state beyond it, two barriers bill twice. At eleven bits that is the band’s error doubled. At three bits it is two draws from a distribution whose median is already a seventh, and the prediction with a sign is that E5M2 loses the third well on most chains at barriers half as deep as those that lose the second.

Rescaling as it goes. Starting high spends the headroom once. A recurrence that multiplies by an exact power of two whenever the running value falls below a threshold spends it repeatedly and never reaches the band, at the cost of one comparison per step and an exponent counter carried beside the vector. In half precision that is the unbounded route; in a byte it would remove every range failure on this page and leave exactly the precision noise of the third figure, which is the measurement that would say how much of E4M3’s survival depth is range and how much is its four bits.

A componentwise certificate for a byte. The earlier essay found that the componentwise backward error sees an empty far well that the normwise one does not. At eight bits the componentwise error of a run whose far well is a seventh wrong is itself a few units of a coarse roundoff, and whether it can distinguish a far well wrong by fifteen per cent from one wrong by half — the only distinction this format permits — is the question that decides whether an eight-bit stationary vector can be shipped with any statement of its accuracy at all.