3.3 Solving Overdetermined Systems

A m × n system of linear equations with m < n is overdetermined. There are more equations than there are unknowns. ”Solving” this equation is the process of reducing the system to an m × m problem then solving the reduced set of equations. A common technique for constructing a reduced set of equations is known as the least squares solution to the equations. The least squares equations are derived by premultiplying Equation 19 by AT, i.e.

(𝐀𝐓𝐀)𝐱=𝐀𝐓𝐛\mathbf{\left({A}^{T}A\right)x={A}^{T}b} (26)

Often Equation 26 is referred to as the normal equations of the linear least squares problem. The least squares terminology refers to the fact that the solution to Equation 26 minimizes the sum of the squares of the differences between the left and right sides of Equation 19.