The search finds a join, not the join

The floor is the part the noise buried found the one way a second penalty paid. The signal is a smooth arch joined to a three-level staircase, deblurred from 64 noisy samples. Uniform mixtures of the zeroth, first and second difference penalties all land on the same floor — the share of the answer the noise has buried — and gain under two per cent over the best single penalty. A pair split in space, the second difference on the rows before the join and the first difference on the rows after it, broke through the floor: by 10 to 31 per cent, on every draw, but only with the join at three quarters, where the arch is long enough to need a strength the steps cannot take. At a quarter and a half it bought nothing. And it was told where the join was.

The essay’s last question was whether it needed telling. “A rule that tries every join position on the 64-point grid and keeps the one with the smallest generalised cross-validation value would cost sixty-four solves; the prediction is that it finds the join to within two grid points on all eight draws at three quarters and keeps at least two thirds of the split pair’s gain, and that at a quarter and a half, where splitting buys nothing, it chooses a position that costs less than a per cent.”

None of those three numbers survives, and two of the reasons are worth more than the numbers.

Fifty-seven joins, each with its own two strengths #

Generalised cross-validation scores a regularised solution by its residual over the square of what the solution leaves free: $n\|b - Ax\|^2 / (n - \operatorname{tr} H)^2$, where $H$ maps the data to the fitted data. A second penalty is not a second parameter was where two penalties first met this rule; here there are two strengths and a position, and the search is exhaustive: at each of 57 trial joins, from the fourth grid point to the sixtieth, both strengths are tried on the same grid of twelve values from $10^{-6}$ to one, 144 pairs, and the pair with the least GCV is kept with its error. The join whose kept pair has the least GCV of all is the search’s answer. That is 8,208 regularised solves a draw rather than the prediction’s sixty-four, because each position needs its own two strengths; in the singular-vector coordinates of the 64-point blur each is a 64 × 64 Cholesky factorisation and its trace, a few seconds a draw.

The picture has three parts and the prediction anticipated none of them. Unguarded, the search collapses on three draws. Floored, it finds the join at three quarters, roughly. And at a quarter it does not look near the join at all.

A collapse on one side #

Unguarded, the search’s picks on three draws — one with the join at a half, two at three quarters — have errors 21.6, 21.5 and 23.6 times the best uniform pair’s. All three pick a first-difference strength of $10^{-4}$ or $3 \cdot 10^{-5}$ on the staircase side, with a reasonable strength on the arch side. That is the classic failure of generalised cross-validation, an undersmoothing whose GCV value is lower than any sensible solution’s because the denominator $n - \operatorname{tr} H$ falls slowly while the residual falls fast; with two penalties it can happen on one side of the join while the other side stays sensible, and the residual on the arch side, well fitted, carries the score.

Floored at a thousandth — the smallest strength the error-minimising pairs use on any of these draws — the collapses disappear and every pick on every draw is within 16 per cent of the best uniform pair. One draw in twenty found GCV with a single penalty giving a catastrophic second answer on four to six draws in a hundred, with its median among the best of five rules; choosing without knowing had found it landing on the oracle’s strength exactly on sixteen. Three collapses in twenty-four is that failure at two to three times its single-penalty rate. With two strengths the chance of a collapse is per side, and a floor on each is the guard the search needs before it can be read at all. Everything below is floored.

Three quarters: the place, not the payment #

With the join at three quarters the floored search picks grid point 45 on five draws, 47 — the join itself — on two, and 39 on one: within two points on seven draws of eight. Its scan on one draw shows why it lands two short so often.

The GCV curve has a broad floor over the last staircase step rather than a notch at the join. Moving the split two points left puts two more rows under the first difference, which on these samples is cheap, and GCV cannot tell the two positions apart by more than its own noise. The error curve agrees on the region and disagrees on the point. The search finds where the join is to within its resolution and not better.

The error curve has its own landmarks, and they are the places a split must not go. On that draw the three worst trial joins are at the 54th, 48th and 52nd points, at 1.8 to 2.7 times the best uniform pair’s error: a cut just after a step puts the step itself under the second difference, which rounds it, and the first difference after it cannot restore the corner. The same pattern holds at every join — with the join at a quarter the worst cuts are at 32, 48 and 21, the staircase’s steps and the arch’s foot, at 2.0 to 2.8 times — and GCV’s curve shows them too, as the spikes the search steps around. A penalty split at a discontinuity is worse than no split at all.

What it does not find is the gain. Told the join and the error-minimising strengths, the split pair’s error on these eight draws is 0.76 to 0.91 times the best uniform pair’s; found by the search, 0.90 to 1.01. The share of the told gain the search keeps has a median of 30 per cent, not two thirds, and on two draws it keeps none.

The figure separates the two losses. At the true join, GCV’s own strengths give 1.00 to 1.31 times the told error, a median of 1.05; the position the search picks costs the rest. Neither alone accounts for the shortfall: two points of position and a GCV-chosen pair of strengths each give back part of a gain that was only 10 to 31 per cent to begin with, and between them they give back two thirds of it. The prediction’s two thirds was the share it expected to keep; it is about the share lost.

A half: a join that buys nothing, found anyway #

With the join at a half, where the told split was within two per cent of the best uniform pair on every draw, the search still finds it: picks at 29 to 31 on six draws of eight, and at 55 and 56 on the other two. Its costs run from 0.95 to 1.16. That is not the prediction’s “less than a per cent”: four draws are more than a per cent better than the uniform pair and one is 16 per cent worse. GCV at the true join with its own strengths is worse than the told strengths by up to a factor of two here — 1.03 to 2.10 — which says that at a half the search’s real difficulty is the strengths, not the place, and that a place it gets right does not save it.

The four draws where GCV’s strengths at the true join cost a factor of 1.7 to 2.1 share one choice: a strength of 0.3 on the arch and the floor itself, a thousandth, on the staircase. That is the collapse again, stopped at the floor rather than prevented — GCV still prefers the least smoothing it is allowed on the stepped side, and the floor only limits how far the preference can go. On the other four draws, where it chose a hundredth or three hundredths on the arch side and three thousandths on the steps, its strengths cost 3 to 17 per cent. So the floor is doing two jobs: it removes the twenty-fold failures, and it caps a bias it does not remove, which at a half is worth a factor of two on half the draws. A floor read from the noise level rather than from the oracle would be the honest version of this guard, and how it would set the cap is the next measurement.

A quarter: the join it was not told about #

With the join at a quarter the search never picks anything near grid point 16. It picks 49 on four draws, and 51, 52, 56 and 59 on the others: every pick past the staircase’s last step, at the 47th point. And on four draws that split beats the best uniform pair by 8 to 15 per cent — where the split told the true join was within two per cent of it.

The picture says what the search found. After the last step the answer is flat, and a flat stretch is exactly what the first difference penalises without bias: the found split reconstructs it with almost no ripple, where the uniform pair’s reconstruction wanders by a few hundredths. Before the cut, the second difference handles the arch and the two lower steps about as well as the uniform pair did. The error falls from 8.0 to 6.8 per cent of the answer’s norm on the draw shown. The signal was built with a join at a quarter, between an arch and a staircase; the join that pays is between a staircase with steps and a staircase without them. Nobody built that one in, and the search found it because it was scoring fits, not reading labels.

Measured against the floor the earlier essay identified, the found split does something no uniform mixture did on this signal. The best uniform pair’s error is 0.91 to 1.05 times the share of the answer the noise buried, which is the floor every uniform mixture, every single penalty and every truncation sat on in four knobs and one floor and the essays after it. The found split’s error is 0.85 to 0.89 times that share on four of the eight draws. It gets under the floor by the mechanism the told split used at three quarters: a first difference on a stretch where the answer is flat recovers coefficients the noise had buried, because it knows something about them — that their combination is constant there — which no uniform penalty can know about one stretch without imposing it on all of them.

A third penalty on a flat floor found that adding a penalty bought nothing on these signals; this finds that moving one bought something on a signal where the told position bought nothing, which is the stronger statement that the structure that matters to a penalty is not always the one in the signal’s description.

Where the shortcut is not #

A search of 8,208 solves a draw invites a cheaper route, and the obvious one fails. Solve once with the best uniform pair and put the join where its residual is largest: on every one of the eight draws the largest five-point residual sits at the same grid point whichever join the signal was built with — the 9th on the first draw, the 4th on the second, the 30th on the fifth — because the data residual of a deconvolution is the noise that the solution declined to fit, and it is largest where the noise happened to be largest. Thirty-two coefficients instead of a noise level read the noise off a residual’s high coefficients for exactly that reason; the same property that makes the residual a good noise estimate makes it a useless locator. Whatever finds the join has to look at the reconstruction, not at what it left behind.

What the prediction asked of a score #

The prediction asked generalised cross-validation to do two jobs at once: locate a structural feature and price two penalties. It does the first roughly, to its resolution, and does it even when the feature buys nothing; it does the second as it always does, with a median loss of a few per cent and an occasional collapse that a floor removes. The prediction’s error was to read the told split’s gain as available to anyone who finds the join. The gain was the join and the strengths together, and the strengths were chosen with the error in hand. A rule that has to be told how good its answer will be found the same dependence in the discrepancy principle; here it is the gain itself that had been told.

There is also the matter of cost. The prediction priced the search at sixty-four solves, one per position, as if the strengths came free. They do not: each position needs its own pair, and a pair chosen at one position is wrong at the next, because moving the join moves which rows each penalty governs. The honest price is positions times strength pairs, here 57 × 144, and for a problem larger than 64 unknowns it is the strength search that would have to be made cheaper first — by a continuation from one position’s best pair to the next, say — before any search over positions is affordable.

What eight draws of one blur do not show #

The blur, the noise level and the signal are the earlier essay’s; a different blur would move the buried share and with it every gain here. The floor at a thousandth is chosen from the error-minimising strengths on these very draws, which is knowledge a user would not have; a floor read off the noise level would be the honest version, and is not measured. The search scores 57 positions with the same 144 strength pairs; a finer strength grid would move GCV’s picks by a little and its collapses by an unknown amount. And the quarter-join gain is four draws of eight at one signal: that a found split beats the uniform pair where the built one did not is shown, and how often it would on other staircases is not.

Still open: a floor from the noise, and a join found by its residual #

A floor read off the noise. The floor that removes the collapse here was taken from the oracle strengths. The prediction with a sign is that a floor set where the penalised solution’s residual first exceeds the noise level by half — a discrepancy-principle floor, computed from the data and the noise level alone — removes all three collapses, changes no floored pick by more than two grid points, and keeps the worst cost within 20 per cent of the best uniform pair at every join.

A join read off the residual is not there. The cheaper search one would hope for — one uniform solve, and the join placed where its residual is largest — was tried on these draws and fails outright: the largest five-point residual of the best uniform pair sits at the 9th, 4th, 17th, 4th, 30th, 47th, 22nd and 49th points on draws one to eight whatever the join is, because a deconvolution’s data residual is dominated by where the noise happens to be large, not by where the reconstruction is worst. So the open question is a different one. The prediction with a sign is that the largest local second difference of the best uniform pair’s reconstruction, not its residual, marks the staircase’s steps on every draw, and that splitting one point past the last such mark reproduces the floored search’s pick to within three points at every join.