Sensitivity and Conditioning (numerical analysis)¶

Definition¶

An inaccurate solution may not be due to the inexactedness of a given algorithm, but can be inherent in the problem being solved. Even with exact computation, the solution to the problem may be highly sensitive to perturbations in the input data.

Thus, we have the qualitative notion of sensitivity and the quantitative notion of conditioning, which focus on the propagated data error. (effects on the solution from perturbed input data).

Sensitivity¶

A problem is said to be insensitive or well-conditioned if a relative change in the input data causes a proportionally relative change in the output.

A problem is sensitive or ill-conditioned if a relative change in the input data causes a significantly larger change in the output.

eg. a sensitive system would be a shower changing from freezing to scalding water from the slightest change to the temperature control.

Example¶

Consider the tangent function, \(f(x) = \tan x\). Since \(f'(x)=\sec^2(x) = 1 + \tan^2(x)\), we have

\[ \text{condition number} \approx \left| \frac{xf'(x)}{f(x)}\right| = \left| \frac{x(1+\tan^2(x))}{\tan x}\right| = \left|x \left(\frac{1}{\tan x}+ \tan x\right)\right| \]

Thus, \(\tan\) becomes highly sensitive to numbers near \(\frac{\pi}{2}\) where its value becomes infinite. For example, take \(x = 1.57079\), the condition number is an insanely large \(2.48275 \times 10^{5}\). This means a relative change in the output is a quarter million times larger than a relative change in the input. Thus, this function is very sensitive at this point.

Condition Number¶

The condition number is defined as

\[ \text{condition number} = \frac{|(\hat{y}-y) / y|}{|(\hat{x} - x) / x|} = \frac{|\Delta y / y|}{|\Delta x / x|}, \]

which is the ratio of the relative change in the solution to the relative change in the input.

The condition number is usually denoted by \(\kappa\), and by rearranging the formula

\[ |\Delta y / y| = \kappa \cdot|\Delta x / x|, \]

where we call \(\kappa\) the "amplification factor" since \(\kappa\) determines the magnitude of the output error given our input error.

Using calculus we can derive an estimate formula for \(\kappa\) to obtain:

\[ \text{condition number} \approx \left|\frac{xf'(x)}{f(x)}\right| \]

Example¶

Consider the function \(f(x) = \sqrt{ x }\). Since \(f'(x) = \frac{1}{2\sqrt{ x }}\), we have

\[ \text{condition number} \approx \left| \frac{xf'(x)}{f(x)} \right| = \left| \frac{x / (2 \sqrt{ x })} {\sqrt{ x }}\right| = \frac{1}{2}. \]

This means that a change in the input data results in a change in the solution with about half that size. Equivalently, the solution error is about half the input error in magnitude.