Linear Systems (numerical analysis)¶
Definition¶
Linear relationships are those whose "effects" are proportional to their "causes". For example, Newtons Second Law of Motion \(F = ma\), says the force is proportional to acceleration, where mass is a constant.
In higher dimensions, such linear transformations are expressed by linear transformations \(\boldsymbol{\mathcal{L}}\) that relates a vector of "causes" \(\boldsymbol{\mathcal{u}}\) to a vector of "effects" \(\boldsymbol{\mathcal{f}}\), so that
In matrix-vector notation, a system of linear algebraic equations is expressed as
where we have a known matrix \(\mathbf{A}\). If we know the "causes" \(\mathbf{x}\), then we can solve for the "effects" \(\mathbf{b}\).
Knowing this equation also allows us to do reverse-engineering. That is, given that we know \(\mathbf{b}\) and \(\mathbf{A}\), we can solve for the "causes" \(\mathbf{x}\).
A system of linear equations \(\mathbf{Ax = b}\) asks the question: "Can the vector \(\mathbf{b}\) be expressed as a linear combination of the columns of \(\mathbf{A}\)?" (ie. "Does \(\mathbf{b}\) lie in \(\text{span}(\mathbf{A}) = \{ \mathbf{Ax}: \mathbf{x} \in \mathbb{R}^{n} \}\)?").
- If so, then the system is consistent, and the coefficients of this linear combination are given by the components in \(\mathbf{x}\).
- There may or may not be a solution; and if there is a solution, it may or may not be unique.
What about Nonlinear Relationships?¶
Even when relationships are non-linear, they can still be approximated locally (ie. for values near a specific value) by a linear relationship.
The derivative does this approximation; think of the tangent line to a nonlinear curve. This observation forms the basis of many numerical methods for solving nonlinear algebraic problems.