The other half of a format

What a float can hold opens this site with the spacing of the representable numbers and the observation that it doubles at every power of two. Every essay since has treated “precision” as meaning the significand, and lib/float.js has simulated the significand and nothing else — each operation performed in double and rounded to a chosen number of significant bits, exactly as IEEE 754 rounds.

That was written down as a limitation on the day it was built, in the file’s own docstring: no overflow to infinity at 10³⁸, no gradual underflow, no subnormals. Everything this site claims is about precision, and precision is what is here.

The limitation had a consequence that grew rather than staying still, and the consequence is a false statement.

The false statement #

lib/mixed.js carries a table of the hardware formats and it is indexed by mantissa bits:

bfloat16: 8 · fp16: 11 · tf32: 11 · fp32: 24 · fp64: 53

Read that table for what it is and it says fp16 and tf32 are the same format. Everything downstream agreed. Where the hardware went reports “fp16/tf32 κ·u = 0.488” as one row, because at eleven bits they compute the same thing — and that row was correct about the computation it described and wrong about the formats it named.

The two are not the same format. tf32’s largest finite number is 3.4·10³⁸ and fp16’s is 65,504, which is thirty-three decades apart, and the whole reason tf32 exists is that difference. It was introduced so that a value which fits in single precision cannot overflow when it is narrowed for a matrix multiplication, and the price paid for it is thirteen significand bits.

So the maturity phase’s job here was not to add a feature. It was to remove a claim the site was making by omission.

A format is two numbers #

An IEEE-style binary format is specified by two integers: the width of the exponent field e, and the precision p — the number of significand bits including the implicit leading one. Everything else follows from those two and from the standard’s rules.

The bias is 2^(e−1) − 1. The top exponent code is reserved for infinities and NaNs, and the bottom one for zero and the subnormals. So the largest finite number is 2^emax·(2 − 2^(1−p)) with emax the bias, the smallest normal is 2^(1−bias), and the smallest subnormal is that divided by 2^(p−1).

Those formulas reproduce the numbers everybody knows, which is the check that entitles this site to say “fp16” rather than “an 11-bit simulation”:

65,504 and 3.4028…·10³⁸ are published constants that came from nowhere near this file, and a formula off by one in the bias reproduces neither. The site checks both, and checks binary32’s largest finite number against Math.fround — the hardware agreeing that the number computed from the formula is representable.

The budget, and why the trade is not symmetric #

The figure is the design decision. A 16-bit format has fifteen bits to split, and every bit given to the exponent is taken from the significand.

What makes the choice interesting is that the two curves have different shapes. A bit of exponent doubles the range, because the range is exponential in the exponent width. A bit of significand adds about a third of a decimal digit, because precision is linear in it.

So the trade is wildly asymmetric, and it explains bfloat16 completely. Going from fp16’s five exponent bits to bfloat16’s eight multiplies the range by 2²⁴ — from a span of about twelve decades to one of about seventy-eight — and costs three significand bits, which is a single decimal digit. For a workload where values span many orders of magnitude and each individual value need only be roughly right, that is not a close call.

It also explains why nobody builds a 16-bit format with twelve exponent bits. The extra range is free in the sense that it costs bits, and useless in the sense that no computation needs 10⁶⁰⁰ — and the three significand bits it costs take the format below two decimal digits.

What the extension does, and what it must not do #

arith(bits) now takes an optional exponent width. With it, the arithmetic context overflows to infinity, underflows gradually, and has a largest and a smallest number. Without it, the behaviour is what it was.

That last clause is the whole engineering constraint. Two phases of figures were drawn against arith(bits) and none of them may move — a change that silently redrew 225 existing figures would be exactly the failure ../CONSOLIDATION.md records about a bad closeGaps breaking 181 figures across four sites.

So it is asserted rather than assumed. Eighteen thousand random values inside fp16’s normal range are rounded by both contexts and required to be bit-identical, and the two are required to differ in exactly the way a format does outside it: 300² is 89,984 without a range, at eleven bits, and is an infinity in fp16.

The default is expBits: null, meaning an unbounded exponent, and the site refuses to let that be described as a format — the assertion that an unbounded context has a range is fed to the build and rejected, because having no largest number is precisely what it was missing.

Where the difference shows up #

Three places, and each gets an essay or a section.

Overflow, which is where a computation leaves the top of the range. This is the fp16-against-tf32 case and it is a norm that overflows before it is a norm: the naive Euclidean norm squares before it adds, which doubles the exponent, so it works over only about half of a format’s range. In fp16 a vector of sixteen entries each equal to 1,000 has a norm of 4,000 — comfortably representable — and the naive expression returns an infinity. In tf32, at the same precision, it returns 4,000.

Underflow, which is the bottom, and it is a different story with a different lesson. That is the numbers below the smallest one.

And the ranking of the formats. Where the hardware went ranks the five by mantissa bits and concludes that bfloat16 is the least accurate. It is, and it is also the only 16-bit format on which a great many computations complete at all — which is a fact about a different axis and which the ranking cannot express.

Why the two axes are not interchangeable #

A reader could reasonably ask why this needs its own axis at all. Both range and precision are ways of failing to represent a number, and a computation that overflows and a computation that rounds have both lost information. The answer is that they fail differently, and the difference is the reason overflow gets a separate treatment everywhere in this subject.

Rounding is bounded and relative. Every operation returns a result within a factor of 1 + u of the exact one. That is what makes backward error analysis possible: a whole sequence of operations can be described as the exact computation on slightly perturbed data, and the perturbation is proportional to the data. Every claim in the error field rests on it.

Overflow is neither. The result is not within any factor of the exact one — it is an infinity, so the relative error is infinite. And infinity is absorbing: infinity minus infinity is a NaN, and a NaN propagates through everything it touches. So a single overflow anywhere in a long computation does not degrade the answer, it destroys it, and there is no analysis that recovers from it.

That asymmetry is why the standard treats them differently. Rounding happens silently on essentially every operation and no sane system reports it. Overflow raises a flag, because it is an event rather than a background condition.

And it is why the fix is different in kind. There is no way to compute more accurately than the format allows; the only response to rounding is a longer significand or a better-conditioned algorithm. But overflow can always be avoided by scaling, at the cost of one extra pass over the data — which is what the next essay’s scaled norm does and what every serious library routine does internally. A failure that can be engineered around and a failure that cannot are different kinds of failure, and collapsing them into “the format was not good enough” loses the only actionable half.

There is one more consequence and it is the practical one. Because rounding is relative, the natural way to think about precision is in significant digits, and the natural way to compare formats is by unit roundoff — which is what every essay on this site did before this one. Because overflow is absolute, the natural way to think about range is in decades of magnitude, and comparing formats by it means comparing 12 against 78. The two comparisons produce different orderings of the same five formats, and there is no single number that reconciles them, which is exactly why the figure above draws two curves and not one.

What is asserted here #

The format constants are the hardware’s. 65,504, 2⁻¹⁴, 2⁻²⁴ for fp16; 3.4028…·10³⁸ and 2⁻¹²⁶ for binary32, with the hardware’s own fround confirming the largest finite number is representable; and binary64’s largest and smallest against the language’s own constants.

fp16 and tf32 have exactly the same unit roundoff, asserted as an equality, and ranges more than 10³³ apart — the pairing this whole extension exists for, stated as two assertions rather than as a sentence.

bfloat16 is the less precise of the two 16-bit formats and the one that cannot overflow where single precision does not. Both halves, because the first alone is the ranking that was already there and the second is what it was missing.

Bounding the exponent changes nothing inside the range, over eighteen thousand values, bit for bit.

The refusals: that fp16 and tf32 compute the same thing, fed a norm that is finite in one and infinite in the other; that overflow is a large rounding error, fed 300² in fp16; and that an unbounded context has a range.

A note on what is still missing #

Three things, and naming them is cheaper than discovering them later.

The fp8 formats are not here. E5M2 follows IEEE’s rules and would fit the machinery exactly; E4M3 does not — it has no infinities and extends its exponent range to reach 448, which the formulas above do not produce. Adding it would mean either special-casing it or reporting a number that is not the one on the hardware, and reporting the wrong number quietly is the failure this site is organised against.

Rounding modes are not simulated. Everything here rounds to nearest, ties to even. Round-toward- zero and the directed modes exist, they change results, and stochastic rounding — which is genuinely used in low-precision training — changes them in a way that matters for exactly the accumulation problems the order they are added in is about.

And the block formats are not here at all. The formats current accelerators are moving to share one exponent across a block of values, which is neither a scalar format nor an array of them, and the whole notion of “the range of the format” needs restating for them. That is a subject rather than an omission.

What is here is the pair of numbers that specifies an ordinary IEEE format, the five formats that pair produces, and the two failures — off the top and off the bottom — that a significand-only simulation could not express.

What this changes about the site’s own claims #

Two of them, and both are corrections rather than additions.

The mixed-precision field’s ranking is now incomplete rather than wrong. Where the hardware went measures iterative refinement at three formats and finds the threshold at κ·u = 1, with bfloat16 failing first because its unit roundoff is largest. Every number in it is correct. What it could not say is that the same experiment run on data spanning more than twelve decades would not produce three results and one failure — it would produce two results, one failure, and one infinity, and the infinity would belong to the format with the second best precision of the three.

And the precision slider means something narrower than it appeared to. Nine figures on this site put the mantissa on a slider and watch a quantity slide with it, which is the site’s precision as the knob thread and is exactly as sound as it ever was — those figures are about the significand and they measure the significand. What they cannot show is a computation failing rather than degrading, and a reader who took the slider as a model of “using a smaller format” was being given half the model. The formats on the slider have no bottom and no top.

Nothing in the existing figures moves. The extension is opt-in, the eighteen thousand-value check above is what guarantees that, and the figures that want the other axis are the three in this essay’s field. But the claim the site was implicitly making — that narrowing a format is a matter of losing digits — was too small, and the correction is the phase’s own subject: a measurement is about the axis it was taken along, and an axis nobody measured is a claim nobody checked.