The general problem formulation

We have a set of data points for which, for example, represent time moments and measurements. We want to find a relationship (function) that gives . For this, we consider a set of predefined functions and claim that the relationship is of the form

where the parameters are unknown for the moment and will be determined based on the data. Usually the number of parameters is much smaller than the number of data pairs,

For example, if and we use polynomial functions , , we recover the previous example, where we fitted a quadratic function; in this example .

To determine we insert the data in the relation () and obtain the following equation

Denoting

the equations become

This system has rows (equations) and only columns (unknowns), with . Therefore we can only find a solution in an approximative sense.