Deconvolution — where it appears
Named by 18 essays across 2 fields — each of them below, with the objects they name alongside it.
When the answer is a choice
A backward-stable least-squares solve of this problem returns an answer whose relative error is 5.5·10⁸. Nothing went wrong. The singular values decay exponentially with no gap anywhere in them, the data does not determine the answer, and something outside the data has to choose — which is the computation rather than a preliminary to it.
A second blur, narrower than the first
A regularised answer is not the truth with the noise taken out. It is the truth seen through a second blur, V F Vᵀ, which depends on the operator and λ and on nothing that was measured. At the best λ for 0.1% noise its rows are 2.82 points wide against the instrument's 5.89, they dip to −0.075 on either side, and their width times the number of components kept stays between 1.10n and 1.27n across seven decades of λ. Two spikes four points apart come back as two; three apart, as one.
The grid was the first filter
A continuous deconvolution discretised on n points and solved with no regularisation at all is not unregularised. Its error against the continuous signal is least at 24, 26 and 34 points for noise of 1%, 0.1% and 0.01% per sample — beside best truncations of 24, 28 and 32 components on a 64-point grid — and within 4 to 16 per cent of their error. The grid's own filter factors sum to n exactly and fall through a half at k = n. Choosing the grid was choosing a truncation, before anybody chose a λ.
A better discretisation is a weaker filter
A coarse grid's error has two sources — how well the discrete operator approximates the integral, and how well the grid's function represents the answer — and the grid essay could not separate them. Changed one at a time they separate: integrating the kernel against the hat functions takes a fifth off the 12-point error, reading the answer as a cubic spline takes 15 per cent more, and both roughly double the condition number on every grid. At 0.1% noise the spline discretisation's unregularised solve on 26 points reaches the best truncation of a 64-point grid to 0.3%. At 1% it is worse than the crude grid.
What a cheap preconditioner has to leave alone
A blur approximated by a matrix the cosine transform diagonalises agrees with the operator everywhere but its first and last seven rows. Made invertible by a shift, as the exact preconditioner was, it never reaches the unpreconditioned run's floor — at α = 10⁻³ its best iterate is 0.749 against 0.143. Made invertible by leaving every eigenvalue below τ alone, it reaches 0.141 in five steps instead of twenty, and the smallest τ that keeps the floor sits at a third to a half of the Tikhonov oracle's λ at three noise levels.
Where the grid hands over to λ
An unregularised solve on a coarse grid comes within a tenth of the best Tikhonov answer on a fine one, and the pair of a grid and a λ was left unmeasured. Measured, the two do not trade. On every grid up to the best unregularised one no λ helps at all. On every grid of 40 points and more the best λ is the same to within a quarter of a decade — 3.2·10⁻² at 1% noise per sample, 10⁻³ at 0.01% — and the 96-point grid with it beats the best coarse grid by 7, 9 and 13 per cent. The grids between the two, given their own λ, land between them.
The grid on which the discretisation stops mattering
Without regularisation, integrating the blur's kernel against cubic splines beat sampling it at 0.1% noise and lost to it at 1%. Give each discretisation its own best λ on every grid and the difference shrinks to nothing where grids are fine — 0.17, 0.28 and 0.20 per cent apart on 96 points at the three noise levels, with every discretisation choosing the same λ — and stays at 17 to 18 per cent on 16 points. The choice between them is a choice of how coarse a grid can be: at 0.1% noise the integrated discretisations reach the fine-grid answer on 26 points and the sampled one needs 40.
The shift had an edge, and the approximation moved it
A fast-transform preconditioner made invertible by a shift was recorded as never reaching the unpreconditioned floor, and predicted to sit off the answer's path at every shift. At a large shift it sits on the path and reaches the floor to a tenth of a per cent. It has an edge like the truncated one — but on the exact operator that edge is where the shift's own effective dimension reaches the answer's, 1.02 to 1.05 of it on six problems, and on the fast approximation it arrives at 0.49 to 0.77. The difference is sixteen samples at the ends of the signal, where the approximation is wrong and a shift divides the error by α.
A corner the penalty can afford
Every smooth reading of the deconvolution's grid needed about forty points and then stopped improving, and the step was the suspect. Give the step one coefficient of its own and forty-eight points reach an error of 0.0070 at 0.1% noise, against 0.118 for the best smooth reading on ninety-six — the step was most of the error. But the same step given two coefficients recovers half as well, and given a doubled node at each edge it recovers worse than no breakpoint at all, while representing the signal to 0.07%. What decides is what the penalty is charged for the corner, and whether the data can say where it is.
One arc, and what each filter pays to be on it
Conjugate gradients and Tikhonov stop at the same effective dimension, and that could have meant two curves crossing once or one curve. It is one curve over a stretch — on six problems, Tikhonov and truncation reach the iteration's error at the iteration's dimension to within 7.2 per cent from 0.7 of the answer to its top — and the two separate on either side. But the curve is shared by a trade, not by an identical answer: at the same dimension Tikhonov carries 22 to 42 per cent more noise than the iteration and up to five per cent less bias.
The data count their dimensions, not the step's
Every grid in the deconvolution essays was chosen with the answer in hand, and so was every λ. From the data alone, the discrepancy principle's worst draw is within 16 per cent of the oracle on every grid from 16 points to 96; generalised cross-validation is better on the median draw and, on grids of thirty points and more, has draws thousands of times worse. And the data can say how many dimensions they carry — about 20, 25 and 29 at three noise levels, one number once the grid exceeds it — but not how many more the step needs: the grid that count chooses is 14 to 19 per cent worse than forty points at the lower two.
The overshoot was the lead
Past its best, conjugate gradients' error rose more slowly than Tikhonov's at the same effective dimension — on the narrow blur at 0.1% noise Tikhonov was 2.35 times worse at 1.3 of the answer — and the reading was that the iteration spends its dimension where the data has content. Count admitted directions instead of summing factors, so that a factor of 1.81 counts once, and the lead is gone: 0.96 on that problem, and within 0.17 of one on all six from 1.1 to 1.3. Tikhonov now carries less noise than the iteration there. Below half the answer, where the three filters also disagree, the count changes nothing.
The minimum on the right
Generalised cross-validation's worst draws on a fine grid were all one mistake: a second dip in its function at λ near zero, deeper than the real minimum. The proposed repair was a residual threshold, one number, refusing any λ whose residual falls too far below the real minimum's. Measured over 528 draws, a threshold of one half still lets two hundredfold misses through; only the extreme value, which is no threshold at all but the rule 'take the rightmost local minimum', removes all eleven. It costs nothing on the coarse grids where the dip is the right answer, and on the collection's own problem over five thousand draws it turns 243 tenfold misses into 17.
A tail from Tikhonov and a corner from truncation
Below half the answer, conjugate gradients beat Tikhonov at a matched count of directions by up to 69 per cent, and the proposed measurement was the sharpness p of a roll-off between Tikhonov and truncation that matches the iteration there. Any p from 2 to 4 closes the gap to a tenth; truncation, the family's limit, reopens it to a third. But no p describes the iteration. Its filter has Tikhonov's slope exactly in its tail and a local sharpness of 2 to 5 on its shoulder, and a single fitted p is a compromise that drifts from 2.3 at the first step to 1.7 at the answer.
More samples take the floor and leave the dip
GCV's catastrophic misses on fine grids were blamed on squareness: on an n × n system the residual and n − t both reach zero as λ does, and their ratio can dip there. With more samples than unknowns neither reaches zero, and the prediction was that the dip would be gone by construction. Half of it is. The minimum at the floor of the scale, 21 draws in 240 on square systems, is gone at every ratio. The interior dip is not — 28, 20, 13, 8 and 4 draws at one to four samples per unknown — and at four per unknown one draw still misses the oracle by 877 times.
The first step's filter, kept for the whole run
Conjugate gradients' first step applies a ramp — Tikhonov's slope up to a corner, then one — and the essay that found it proposed the ramp as a fixed method, predicting that it would match the iteration below half the answer better than any sharpened Tikhonov and lose the match near the answer. The first half holds by more than predicted: 0.044 from the iteration below half, where the best member of the family was 0.106. The second does not: near the answer the ramp is within 0.016, closer than any filter measured. And two numbers read off the iteration's own Ritz values — its tail's slope and its smallest Ritz value — reproduce it past the top as well, where every fixed filter falls ten to seventeen per cent below it.
A corner chosen without the iteration
The ramp — conjugate gradients' first-step filter kept as a fixed method — matches the iteration's error at the same position, but every comparison placed it at the iterate's position, which needs the iteration. Placed instead by the iteration's own stopping rules, it was predicted to land within the same tenth of its best and earlier. The discrepancy principle does exactly that, earlier on 61 draws of 72 and dearer by a few hundredths on 71. The L-curve finds the ramp's corner as well as Tikhonov's and far better than the iteration's: twice the best on 26 draws against 44, worst 16 against 896. And GCV fails the ramp in a way it never fails the iteration — choosing a solution that discards three quarters of one direction, 696 times its best — until its first minimum from the regularised end is taken, after which the ramp is within a tenth of its best on 64 draws of 72 and never worse than 1.25.
A power the spectrum does not set
Conjugate gradients' filter past the answer is a ramp raised to a power m between 1.27 and 1.50, read off its own Ritz values, and the power was predicted to be the blur's: computable from the decay of the singular values to within 0.1, so the iteration's filter could be built without running it. It is not. Summed over the singular values above the iteration's own corner, the decay gives anything from 0.76 to 4.3, because the Ritz values are not those singular values — fewer of them early in the run, each standing for a cluster, and more of them late, the leading ones found again and again. The power is the spacing between the smallest Ritz value and its two neighbours. And nothing depends on it: placed by GCV's first minimum, the power ramp at every m from 1 to 3 lands within a tenth of its best on 63 to 66 draws of 72.
Named alongside it
The objects these essays reach for when they reach for this one.
Filter factorsTikhonov regularisationRegularisationTruncated SVDConjugate gradientsDiscretisationIterative regularisationEffective dimensionIll-posed problemDiscrepancy principleGeneralised cross-validationSemi-convergence