The digit that costs more than the tensor

A compression is a comparison, and this collection has spent most of its attention on one side of it. The essay that built the train counts what the format stores — the sum of rₖ₋₁ nₖ rₖ over the cores — and watches that count rise as the tolerance is tightened. The other side of the comparison, the array’s own nᵈ entries, has been present in every one of those measurements and has never been near enough to matter. At five indices and eight points a side the tightest tolerance in the sweep costs 2,504 numbers against 32,768 entries, which is a ratio of thirteen and not a question.

Move the same sweep to three indices and six points a side and the second number stops being scenery. The array is 216 entries. The train is 90 numbers at two digits, 210 at six, and 288 at seven — which is thirty-three per cent more than the thing it is representing, computed correctly, to the tolerance asked for, with no fault to report anywhere.

The interesting quantity is therefore not the rank and not the storage but the point where the two curves cross, and where that point sits is decided by the grid and by the number of indices rather than by anything about the function. The measurements below put it at 10⁻⁷, 10⁻⁹, 10⁻¹² and 10⁻¹⁴ for grids of six, eight, ten and twelve points, and nowhere at all once a fourth index is added.

Six points a side, and the array wins after six digits #

The family is 1/(1 + i₁ + … + i_d), which is the one this field uses when it wants a function that is smooth, genuinely low rank at loose tolerances, and not a theorem. Its train ranks are measured rather than written down, unlike the sine family whose ranks are two at every cut by the addition formula, and that is exactly what makes it the right object here: a rank that is a decision taken at a threshold is a rank that moves when the threshold moves.

On the 6 × 6 × 6 grid it moves like this. At 10⁻² both cuts have rank three and the train is 90 numbers. At 10⁻³ and 10⁻⁴ they are four and it is 144. At 10⁻⁵ and 10⁻⁶ they are five and it is 210. At 10⁻⁷ they are six, and it is 288.

Two hundred and ten is the last storage in that list that is smaller than 216. The margin is six numbers — under three per cent — and one more decade of accuracy spends it and thirty-three per cent more besides. So the honest description of the format on this array is that it pays for six digits and not for seven, and that the failure when it comes is not gradual: the storage goes from 97 per cent of the array to 133 per cent in a single step, because a rank is an integer and the step from five to six is the whole of it.

Nothing about that step is a defect of the decomposition. Every train in the sweep meets its stated tolerance, and the measured errors are 1.2·10⁻³, 4.2·10⁻⁵, 4.2·10⁻⁵, 8.3·10⁻⁷ and then 1.2·10⁻¹⁵ onwards. The format did what it was asked. What it was asked for stopped being worth asking.

The break-even rank is n − 1, and it fits with n numbers to spare #

Eight points a side moves the crossing but not the shape of it, and the two grids together are enough to say what the shape is.

At three indices there are two cuts. The first puts one index on the rows and two on the columns, so its rank cannot exceed min(n, n²) = n; the second is its mirror and cannot exceed n either. The unfolding itself imposes the ceiling, before any question of structure arises, and it is the same n both times. So a three-index train with both ranks equal to r holds three cores of shapes (1, n, r), (r, n, r) and (r, n, 1), which is

n·r² + 2·n·r numbers, against n³ entries.

The format pays while n·r² + 2nr ≤ n³, which is r² + 2r ≤ n², which is r ≤ √(n² + 1) − 1. For every integer n that is r ≤ n − 1. The break-even rank at three indices is exactly one below the grid, whatever the grid is and whatever the function is.

The margin at that rank is the part worth carrying, because it is not approximately anything. Substituting r = n − 1 gives n³ − n(n − 1)² − 2n(n − 1) = n, exactly. A three-index train sitting at its break-even rank fits inside the array with n numbers to spare, and the four grids measured agree to the unit: 216 − 210 = 6, 512 − 504 = 8, 1,000 − 990 = 10, 1,728 − 1,716 = 12.

The two terms in that count are worth separating, because they are doing different jobs. The interior core, n·r², is what replaces the array: it is the compression, and it is small exactly when r is well below n. The two end cores, 2·n·r between them, are not compression at all — they are the bases the interior core is expressed in, and they are pure addition. So a three-index train pays only while the saving on the first term is large enough to cover the second, and since the second grows linearly in r while the first grows quadratically, the two meet once and never again. The single crossing is a property of that arithmetic and not of any array.

The right-hand column is the other closed form and it says something the ratio alone does not. At the ceiling rank the storage is n³ + 2n², so the excess over the array is 2n² — which as a fraction is 2/n, and therefore falls as the grid is refined. A finer grid does not make the failure worse. It makes the failure milder and pushes it further down the tolerance axis, which is a different consolation from the one a reader expects and is the reason the crossing has to be measured rather than reasoned about.

The price of a digit does not depend on the grid, and the badge says it does #

The badge on each of these figures reports the exchange rate — how much rank a decade of accuracy buys — and across the four grids it reads 0.30, 0.50, 0.70 and 0.80. Read as it stands, that says the cost of a digit nearly triples between a coarse grid and a fine one, which would be a substantial fact about the format and would sit oddly beside the hierarchy field’s measurement of 0.55 columns a decade for a kernel block, a number that did not move with anything.

It is not a fact about the format. Each badge is an endpoint slope over a fixed window of ten decades, from 10⁻² to 10⁻¹², and on the coarser grids half that window is spent against a ceiling. At n = 6 the rank reaches six at 10⁻⁷ and stays there for five more decades, because six is the largest rank the cut admits and the train at that rank reproduces the array exactly. Three of the seven points on that curve are the same point.

Measured only over the tolerances where the truncation is still discarding something — where the error is above the rounding level and the rank is genuinely being bought — the same four grids give 0.50, 0.68, 0.67 and 0.75 of rank per decade, by least squares over five, seven, ten and ten points. The spread falls from a factor of 2.7 to a factor of 1.5, and what is left of it is largely the n = 6 window being half the length of the others and truncating at rank five.

So the exchange rate is close to two thirds of a rank per decade on all four grids, and the badge’s apparent variation is the ceiling showing through. What varies with the grid is not the price of a digit. It is how many digits can be bought before the rank runs out of room — which is the quantity the crossing is about, and the reason the two look like the same measurement and are not.

Half of a sweep on a coarse grid is exact arithmetic, not compression #

The n = 6 curve’s last three points deserve naming for their own sake, because a figure that repeats a value three times is easy to read as a plateau in the thing being measured.

At 10⁻⁷ and everything tighter, the measured error of that train is 1.2·10⁻¹⁵ — the rounding level of the arithmetic, not the tolerance that was asked for. Both ranks are at their ceiling of six, no singular value has been discarded, and the three cores multiply back to the array to fourteen digits. The train is no longer an approximation of the tensor at all; it is a rewriting of it in 288 numbers where 216 were enough.

That is a genuinely different regime from the one the first four points are in, and it is the regime in which “the format meets its tolerance” stops being informative. An exact ground truth is the strongest kind of check this site has, and it is worthless as a measurement of cost: the arithmetic is being asked whether it can reproduce a small array from a full-rank factorisation of it, and of course it can.

The same regime arrives at 10⁻⁹ on the eight-point grid and at 10⁻¹² on the ten-point one, and does not arrive at all within the sweep at twelve points, where the tightest tolerance still leaves the rank at eleven of a possible twelve. So the four figures are not four instances of one curve. They are four different fractions of one curve, and the fraction that is informative grows with the grid.

The finer sweep, taken decade by decade rather than at the seven tolerances the figures draw, puts the crossing at 10⁻⁷, 10⁻⁹, 10⁻¹² and 10⁻¹⁴. Seven decades of grace at six points a side and fourteen at twelve: the number of digits the format is worth paying for roughly doubles as the grid does, which is the useful form of the result and is not visible on any single figure.

What decides it is the number of indices, not the tensor #

The fourth index is where the argument turns, and it turns harder than the storage counts alone suggest.

At four indices and eight points, the same family at the same seven tolerances runs from 192 numbers to 1,536, against 4,096 entries. The tightest point in the sweep is 37 per cent of the array, and carried past the figure’s range the storage reaches 1,792 at 10⁻¹⁶ and still does not cross. The crossing that arrives after seven decades at three indices does not exist at four.

The tempting reading is that the four-index array is more compressible. It is not, and the ranks say so directly. At 10⁻¹² the four-index train’s cut ranks are 8, 11, 8 — and the outer two are at their ceiling, exactly as the three-index train’s both were. The first cut of a four-index array on eight points has min(n, n³) = 8 rows’ worth of room and uses all of it; the array is as incompressible across that cut as anything can be. The middle cut has room for n² = 64 and uses eleven, which is where all of the compression lives.

The four-index format has a ceiling of its own, and it is instructive that nothing here comes near it. At the maximum ranks the cuts admit — n, n², n — the storage is 2n⁴ + 2n², which at eight points is 8,320 against 4,096 entries, so a fully incompressible four-index array is stored at twice the size of itself rather than at 1.33 times. The failure is worse when it happens. It is simply much further away: holding the outer ranks at their measured ceiling of eight, the storage is 128 + 128·m in the middle rank m, so the format pays while m ≤ 30 and hits 4,096 exactly at m = 31. The measured middle rank is eleven at 10⁻¹² and thirteen at 10⁻¹⁶, which is under half the budget with nothing left to spend on the other side.

So the same function, discretised the same way, sampled at the same tolerance, presents identical end-cut behaviour at three indices and at four. What changed is the number it is being charged against: 512 against 4,096, a factor of eight for one more index, while the train’s own cost grew by a factor of 2.4. That is the curse working in the format’s favour rather than against it, and it is the whole of why the format is used where it is used.

It also fixes what a warning about this should say. The hazard is not tensors that fail to compress — it is small arrays, where the format’s overhead is charged against a budget that never got large. Three indices is where it is marginal; two would be a matrix and a plain truncated decomposition would settle it; four and above is where the comparison stops being close. The same shape appears in the hierarchy field, where a format’s crossover against a dense factorisation sits between 64 and 128 unknowns and walks right when the accuracy is tightened. Both are measurements of a size below which structure is not worth exploiting, and neither is a statement about the structure.

The same 288 that noise costs #

The last measurement is the one that closes the argument, and it is a coincidence of two numbers that is not a coincidence.

On the 6 × 6 × 6 grid at 10⁻⁸, an array of independent normal entries has cut ranks 6 and 6 and stores 288 numbers. At the same size and the same tolerance the reciprocal family has cut ranks 6 and 6 and stores 288 numbers. To the unit, the smooth function with a decaying spectrum and the array with no structure whatsoever cost the same, and both are represented exactly.

They agree because both have reached the ceiling, and the ceiling does not care why. A rank-revealing construction asked for more accuracy than the structure can supply returns the largest rank the cut admits, honestly, in both cases — and once two objects are both at the ceiling there is nothing left to distinguish them by storage. The noise array is at the ceiling from the first decade; the reciprocal array walks there over five. The destination is identical.

That is what the site’s refusal for this page is stated about. The claim under test is the natural one: that a format built out of rank-revealing decompositions can never store more than the array, since in the worst case it will find no structure and fall back on the entries. It does not fall back on the entries. It stores the entries plus the two end cores, which is 2n² numbers of bookkeeping laid on top of n³, and the assertion that a noise train fits in a quarter of its array is fed a 6 × 6 × 6 case and fails by a factor of more than five.

The neighbouring essay reads that refusal as a statement about incompressible objects, and it is one: a random array has no train ranks and is stored badly for a reason nobody could object to. This page reads it the other way. The 288 is not a property of noise. It is the price of the ceiling, and any tensor at all will pay it once the tolerance is tight enough — a smooth one simply takes longer to get there. The same trap has already been recorded one field over, where a renumbered kernel matrix goes from 27,008 numbers to 118,208 with its spectrum unmoved, and where a kernel with nothing to compress is the control every claim of this kind needs.

What follows for anything that stores one #

The consequences are narrow and each of them is a number rather than an attitude.

Compare the two counts before storing anything. The check is Σₖ rₖ₋₁ nₖ rₖ against Πₖ nₖ, it costs one multiplication per core, and it turns a silent thirty-three per cent overhead into a decision. The building essay proposes it as the defence a train lacks, and these measurements say what it would catch: not exotic arrays, but ordinary smooth ones at ordinary tolerances on small grids.

Do not quote an exchange rate off an endpoint. The 0.30 a decade this page begins with is a true number about a window and a false one about the format, and the difference is five decades of saturation hidden inside it. The rate that transfers is the one measured where the truncation is still discarding, and it is close to two thirds of a rank a decade on every grid here. That is a worse habit to have than it looks, because it fails in the flattering direction: an endpoint slope taken across a ceiling always understates what a digit costs.

Expect the price of a digit to be flat, and be suspicious when it is not. Two thirds of a rank a decade here, 0.55 columns a decade for a kernel block, and half a Kronecker term a decade for an inverse: three objects, three fields, one shape of answer, because all three are versions of a smooth function of separated arguments being nearly separable. The one method on this site whose exchange rate improves as it is paid, where each doubling of work buys more digits than the last, is worth knowing about precisely because it is the exception.

A rank that does not move is a different claim from a rank that is small. The measurement next door shows a well-separated kernel block needing five columns at four sampling densities — a rank that does not notice the grid at all. The trains here do notice it, on both axes: the rank at a fixed tolerance rises with n, and the tolerance at which the rank reaches its ceiling falls. The two behaviours look alike on a single figure and are opposite as engineering.

And a tolerance in a solver is not a tolerance in a decomposition. Everything above prices a single decomposition of a fixed array. The moment the format is iterated in, the tolerance stops being a knob a caller turns and becomes a budget the memory fixes, the ranks of every intermediate grow faster than the ranks of the answer, and the relevant question is no longer whether the representation fits but whether what it converges to is worth reaching. The comparison this page makes — a count of stored numbers against a count of entries — is the same comparison a block with no zero entries forces at the other end of the collection, and in both places it is the only one that decides anything.