Numerical Linear Algebra
where the algebra stops being the arithmetic
Concept

Parameter choice — where it appears

The problem of picking a regularisation parameter from the data alone, which no rule solves exactly because the best value needs the answer.

Named by 2 essays across 2 fields — each of them below, with the objects they name alongside it.

00.250.50.75110⁻¹110¹10²10³fraction of the method's own rangerelative errorfloor 0.141truncation KTikhonov λCGLS steprandomised rankfour methods, one floortruncation K0.14Tikhonov λ0.14CGLS step0.14randomised rank0.14four knobs from four fieldsand one obstruction underneath them

Four knobs and one floor

A truncation, a Tikhonov parameter, a step count and a randomised rank, on one problem with an answer that is known. Their best errors are 0.1445, 0.1406, 0.1426 and 0.1449 — a spread of 3% across four methods that share no arithmetic.

combination · parameter choice 10⁻²10⁻¹110¹10²10³10⁴‖Ax − b‖‖x‖the oraclediscrepancyL-curvegeneralisedscored against a truth none hasoracle, relative error0.11discrepancy principle, as a multiple1.1L-curve corner, as a multiple2.3generalised cross-validation, as a multiple1the oracle needs the exact answer and is not a methodit is the reference the others are scored on

Choosing without knowing

Three published rules for choosing a regularisation parameter, scored against an oracle that requires the exact answer and is therefore not a method. Generalised cross-validation lands on the oracle's λ exactly; the discrepancy principle costs 6%; the L-curve costs 129%. And told a noise level ten times too small, the discrepancy principle's error goes from 0.112 to 10,449.

regularisation · parameter choice

Named alongside it

The objects these essays reach for when they reach for this one.

Ill posed problemTikhonov regularisationCross validationDiscrepancy principleFilter factorsIterative regularisationL curveNoise floorRandomised SVDRegularisationSemi convergenceSingular values

All concepts