Eight bits, and a format that breaks the rules

The other half of a format established that a floating-point format has two parameters and that reading either alone gets a real question wrong — fp16 and tf32 have identical precision and ranges twenty-three decades apart, and a site that simulated the significand alone was obliged to call them the same format.

At eight bits the two parameters stop being separate choices. There are seven bits to divide between them, and the two formats that hardware offers divide them the only two ways that make sense:

Neither dominates. E5M2’s steps are exactly twice as coarse and its range is exactly 128 times wider, so a computation has to choose, and the choice is usually made per tensor rather than per program: the values in a network in one format, the gradients in the other.

And one of them is not an IEEE format.

What E4M3 does that IEEE forbids #

Every format in formatRange up to this point follows the same two reservations. The top exponent code means infinity or NaN, and the bottom means zero or a subnormal. Every constant this site prints — 65504 for fp16, 3.4028…·10³⁸ for binary32 — follows from those two sentences and a bias of 2^(e−1) − 1.

E4M3 keeps the second and breaks the first. The all-ones exponent code carries ordinary finite values, and only the single pattern that has an all-ones significand as well is reserved, for NaN. So there is no infinity in the format at all, and the largest finite value is

2⁸ × 1.75  =  448

where IEEE’s reservation on the same four exponent bits and three significand bits would give

2⁷ × 1.875  =  240

The departure is worth 1.87× of range, which is an octave and a bit, and its price is every infinity the format might have had.

Both numbers are checked against published constants that came from a specification rather than from this site’s arithmetic — 448 and 57344, 2⁻⁹ and 2⁻¹⁶ — and a bias off by one, or a reservation applied in the wrong place, reproduces none of them.

Why an octave was worth it #

The trade looks strange from a distance. Giving up infinities to gain a factor of 1.87 in range is not obviously a good bargain when the format above it, E5M2, has 128 times the range for free.

It stops looking strange once the question is asked at eight bits rather than at fifty-three. E4M3’s range runs from 2⁻⁹ to 448, which is about seventeen octaves in total. One extra octave is 6% of the whole dynamic range of the format, bought for an encoding that, in the workloads these formats exist for, would have been a fatal error anyway. A network activation that overflows fp8 has already lost; propagating an infinity through the rest of the computation is not a better outcome than saturating, and no code was going to check for it.

That is the reasoning, and it is a reasonable one. What makes it worth an essay is what it leaves undefined.

Overflow, with nothing to overflow to #

IEEE 754’s rule for a value above the largest finite number is exact: in round-to-nearest it becomes an infinity, and in a directed mode that was rounding toward the finite range it becomes the largest finite number. This site’s roundToFormat implements both, and the second is the reason interval arithmetic still has a lower bound when its upper bound has overflowed.

E4M3 has no infinity, so the first rule has no target. The specification leaves the choice to the implementation, and the two conforming answers are:

The same computation therefore returns 448 on one conforming device and NaN on another, and neither is wrong. That is not a subtlety about a corner case; it is a difference between a number and a poison value, propagating through everything downstream.

So saturate is a real parameter in this site’s arithmetic rather than a convenience, and the default is the signalling one, on the grounds that a NaN is at least visible. The measurement runs both: rounding 500 to E4M3 gives NaN by default and 448 with saturation, and both are asserted.

There is a second-order consequence worth stating. Under saturation the format is no longer monotone in the way IEEE guarantees — 500 and 10⁶ both become 448, so information about how far past the top a value went is destroyed rather than flagged. Under signalling it is preserved as “somewhere past the top”, which is less than a magnitude and more than nothing.

What each format actually holds #

The formats are small enough to enumerate, which no other format on this site is, so the figure above draws every value rather than sampling.

E4M3: 126 positive finite values. Seven subnormals, spaced 2⁻⁹ apart from 2⁻⁹ to 7·2⁻⁹; then fifteen octaves of eight values each, from 2⁻⁶ up, with the top octave truncated at 448. With their negatives, two zeros and two NaN patterns that is 256 — the whole byte, accounted for.

E5M2: 123 positive finite values. Three subnormals; thirty octaves of four values each, from 2⁻¹⁴ to 57344. With their negatives, two zeros, two infinities and six NaN patterns, 256 again.

Two things are visible in that accounting. E5M2 spends eight of its 256 patterns on non-numbers where E4M3 spends two, which is a third of the difference in their value counts. And neither format’s set of values is a subset of the other’s: E5M2 reaches both further out and further in, and E4M3 is finer everywhere they overlap.

The subnormals carry a real share of the range #

The numbers below the smallest one argues that gradual underflow is what makes x − y = 0 mean x = y, and measures the subnormal band at p − 1 octaves — a property of the significand rather than of the exponent field. At fifty-three bits that band is 52 octaves out of about 2,100, which is 2% of the format’s range and is easy to think of as a corner.

At eight bits it is not a corner. E4M3’s normals run from 2⁻⁶ to 448 — about seventeen octaves — and its subnormals add three more below, from 2⁻⁹ to 2⁻⁷. That is 15% of the format’s whole dynamic range, carried by seven values.

Those seven values are also where the precision collapses. A subnormal has fewer significant bits than a normal, by exactly how far below 2^emin it sits, so 2⁻⁹ — E4M3’s smallest positive number — has one significant bit. It is representable and it is representable to within a factor of two.

The practical consequence is that flush-to-zero, which several accelerators do by default and which the subnormal essay measures the cost of, is a much larger intervention here than at higher precisions. Turning it on removes 15% of E4M3’s range and takes its smallest positive number from 0.00195 to 0.0156 — a factor of eight, on a format whose values are already only a factor of 1.06 apart at the top. Whether that matters depends entirely on what is being stored, and the point is that it is a decision of the same size as the choice between the two formats, made by a hardware default rather than by a program.

E5M2’s subnormals are three values covering two octaves out of about thirty, which is a smaller share — the format that spent a bit on range has less of its range down there in the first place.

What a real number looks like in each #

One worked value, because the abstractions above are easier to trust with an instance.

Take 0.3. In E4M3 the nearest values either side are 0.28125 and 0.3125 — the octave [0.25, 0.5) is divided into eight — so 0.3 becomes 0.3125, a relative error of 4.2%. In E5M2 the same octave is divided into four, the neighbours are 0.25 and 0.3125, and 0.3 becomes 0.3125 as well, this time by a narrower margin: it is 0.05 above one neighbour and 0.0125 below the other.

Take 300. E4M3 holds 288 and 320, so 300 becomes 288 — a relative error of 4%, and the value is inside the octave IEEE would have reserved. E5M2 holds 256 and 320, so 300 becomes 320, a relative error of 6.7%.

Take 30000. E4M3 cannot hold it at all: it is past 448, so the answer is NaN or 448 depending on the device. E5M2 holds 28672 and 32768, and 30000 becomes 28672, a relative error of 4.4%.

Three values, and the pattern is the whole trade in miniature: below the point where E4M3 runs out, E4M3 is about twice as accurate; above it, E4M3 has no answer at all and E5M2 has one that is a few percent out. Which format is right depends on where the data is, which is why the choice is made per tensor rather than once for a program.

Why the exponent is the expensive half #

One arithmetic observation, which is where the next essay starts.

At fifty-three bits the exponent field costs eleven bits out of sixty-four — 17% of the storage — and nobody thinks about it. At eight bits it costs four or five out of eight, which is more than half the format, and it is buying something the significand cannot: the ability of each individual value to sit anywhere in a range of seventeen or thirty octaves.

That ability is worth its price when the values genuinely span such a range. It is worth nothing when they do not. Thirty-two activations drawn from the same layer of the same network are usually within a factor of a hundred of each other — seven octaves — and each of them is carrying four exponent bits capable of describing seventeen, which is four bits spent describing a range the data does not occupy.

That is a compression argument rather than a numerical one, and it is the observation the whole microscaling family is built on: spend the exponent once for a group, keep a small integer per value, and the per-value cost drops from eight bits to six and a quarter. Whether the resulting format is any good is a question about the group’s internal range rather than about its bit count, and the next essay measures it.

The unit roundoff, and what it is not enough of #

E4M3’s unit roundoff is 2⁻⁴ = 0.0625 and E5M2’s is 2⁻³ = 0.125. Those are the numbers every error bound on this site is stated in, and at these widths they are large enough to make the bounds read oddly.

A dot product of length n has a backward error bound of about nu, so at E4M3 a dot product of seventeen terms has a bound of greater than one — the bound has stopped constraining anything. Mixed precision exists for exactly this: the products are formed in fp8, the accumulation is done in fp32 or wider, and the bound that matters is the accumulator’s. An fp8 matrix multiply on current hardware does not accumulate in fp8, and reading its error behaviour off 2⁻⁴ would be reading the wrong number.

That is worth stating plainly because it is where the site’s own knob has to be turned carefully. The precision slider on the earlier essays models arithmetic performed entirely at the stated width, which is the honest simulation of a uniform-precision computation and is not what an eight-bit format is used for. The fp8 formats are storage and input formats. What is measured here is what they can hold, and holding is what the block-format essay goes on to compare them at.

What the site’s own machinery had to grow #

Two changes, and the first is small enough to describe and large enough to matter.

formatRange computes every constant from the bias and the two reservations, and it now takes a flag. With noInfinities, the top usable exponent is the bias plus one rather than the bias, and the largest significand is one step lower because a pattern is kept for NaN. Six lines, and the whole of E4M3’s departure lives in them — which is the right size for it, since the departure is a change to one rule rather than to a family.

roundToFormat grew the harder half. Past the top of the range it now distinguishes three cases: a directed mode rounding towards the finite range returns the largest finite number, which is IEEE’s rule and the reason a bracket survives an overflow at one end; a format with no infinities returns NaN, or the largest finite value if the caller asked for saturation; and everything else returns an infinity. Before this phase the function had one case.

Both changes are additions rather than edits, and the regression test is the one the fleet’s rules ask for: the whole site was rebuilt against them and not one of the 46 existing essays changed by a byte. That is what entitles a phase to extend an arithmetic four other phases have been measuring through.

What is asserted, and what refuses #

Both formats’ constants are the published ones: 448 and 57344, 2⁻⁶ and 2⁻¹⁴, 2⁻⁹ and 2⁻¹⁶.

The departure is worth 1.87×, asserted as a ratio against the same bits under IEEE’s rules, which are computed by the same function with one flag changed.

E5M2 keeps the reservation and E4M3 does not, asserted as a property of the specification table rather than inferred from the numbers.

Neither dominates: the ratio of unit roundoffs is exactly 2 and the ratio of ranges exactly 128, both asserted as equalities rather than approximations.

Overflow has two conforming answers, both computed: NaN by default, 448 under saturation.

Every value drawn is one the format holds exactly, checked by round-tripping all 126 of them through roundToFormat with a worst error of exactly zero — which is what entitles the figure to say it is drawing the format rather than a sample of it.

And E4M3 read as an IEEE format is refused. The assertion that IEEE’s rules reach 448 is fed to rejects(); it computes 240 and fails. Every formula on this site would have given that answer, and the refusal is what stops the departure from being quietly re-absorbed.

What is left #

The arithmetic, as opposed to the storage. Nothing here runs a decomposition at eight bits, because a decomposition at eight bits with eight-bit accumulation is not something any hardware does and simulating it would be measuring a machine nobody built. What the mixed-precision field already measures — a low-precision factorisation with a higher-precision residual — is the honest version of the question, and it stops at bfloat16.

The other eight-bit formats. There are more than two: variants with different bias offsets, and the ones that trade the subnormals away entirely. This essay measures the pair the microscaling specifications settled on.

And sharing the exponent, which is the next question and the one that makes the accounting above look generous: if the exponent is the expensive half, why give each value its own?