Approximation of experimental data. Least square method. Where is the least squares method used? The least squares method for a line in space

The method of least squares (OLS) allows you to estimate various quantities using the results of many measurements containing random errors.

Characteristics of MNEs

The main idea of ​​this method is that the sum of squared errors is considered as a criterion for the accuracy of solving the problem, which they strive to minimize. When using this method, both numerical and analytical approaches can be used.

In particular, as a numerical implementation, the least squares method implies taking as many measurements as possible of the unknown random variable. Moreover, the more calculations, the more accurate the solution will be. Based on this set of calculations (initial data), another set of estimated solutions is obtained, from which the best one is then selected. If the set of solutions is parameterized, then the least squares method will be reduced to finding the optimal value of the parameters.

As an analytical approach to the implementation of LSM on a set of initial data (measurements) and an expected set of solutions, a certain one (functional) is determined, which can be expressed by a formula obtained as a certain hypothesis that requires confirmation. In this case, the least squares method comes down to finding the minimum of this functional on the set of squared errors of the original data.

Please note that it is not the errors themselves, but the squares of the errors. Why? The fact is that often deviations of measurements from the exact value are both positive and negative. When determining the average, simple summation may lead to an incorrect conclusion about the quality of the estimate, since the cancellation of positive and negative values ​​will reduce the power of sampling multiple measurements. And, consequently, the accuracy of the assessment.

To prevent this from happening, the squared deviations are summed up. Even moreover, in order to equalize the dimension of the measured value and the final estimate, the sum of the squared errors is extracted

Some MNC applications

MNC is widely used in various fields. For example, in probability theory and mathematical statistics, the method is used to determine such a characteristic of a random variable as the standard deviation, which determines the width of the range of values ​​of the random variable.

Approximation of experimental data is a method based on replacing experimentally obtained data with an analytical function that most closely passes or coincides at nodal points with the original values ​​(data obtained during an experiment or experiment). Currently, there are two ways to define an analytical function:

By constructing an n-degree interpolation polynomial that passes directly through all points a given data array. In this case, the approximating function is presented in the form of: an interpolation polynomial in Lagrange form or an interpolation polynomial in Newton form.

By constructing an n-degree approximating polynomial that passes in the immediate vicinity of points from a given data array. Thus, the approximating function smooths out all random noise (or errors) that may arise during the experiment: the measured values ​​during the experiment depend on random factors that fluctuate according to their own random laws (measurement or instrument errors, inaccuracy or experimental errors). In this case, the approximating function is determined using the least squares method.

Least square method(in the English-language literature Ordinary Least Squares, OLS) is a mathematical method based on determining the approximating function, which is constructed in the closest proximity to points from a given array of experimental data. The closeness of the original and approximating functions F(x) is determined by a numerical measure, namely: the sum of squared deviations of the experimental data from the approximating curve F(x) should be the smallest.

Approximating curve constructed using the least squares method

The least squares method is used:

To solve overdetermined systems of equations when the number of equations exceeds the number of unknowns;

To find a solution in the case of ordinary (not overdetermined) nonlinear systems of equations;

To approximate point values ​​with some approximating function.

The approximating function using the least squares method is determined from the condition of the minimum sum of squared deviations of the calculated approximating function from a given array of experimental data. This criterion of the least squares method is written as the following expression:

The values ​​of the calculated approximating function at the nodal points,

A given array of experimental data at nodal points.

The quadratic criterion has a number of “good” properties, such as differentiability, providing a unique solution to the approximation problem with polynomial approximating functions.

Depending on the conditions of the problem, the approximating function is a polynomial of degree m

The degree of the approximating function does not depend on the number of nodal points, but its dimension must always be less than the dimension (number of points) of a given experimental data array.

∙ If the degree of the approximating function is m=1, then we approximate the tabular function with a straight line (linear regression).

∙ If the degree of the approximating function is m=2, then we approximate the table function with a quadratic parabola (quadratic approximation).

∙ If the degree of the approximating function is m=3, then we approximate the table function with a cubic parabola (cubic approximation).

In the general case, when it is necessary to construct an approximating polynomial of degree m for given table values, the condition for the minimum of the sum of squared deviations over all nodal points is rewritten in the following form:

- unknown coefficients of the approximating polynomial of degree m;

The number of table values ​​specified.

A necessary condition for the existence of a minimum of a function is the equality to zero of its partial derivatives with respect to unknown variables . As a result, we obtain the following system of equations:

Let's transform the resulting linear system equations: open the brackets and move the free terms to the right side of the expression. As a result, the resulting system of linear algebraic expressions will be written in the following form:

This system of linear algebraic expressions can be rewritten in matrix form:

The result was a system linear equations dimension m+1, which consists of m+1 unknowns. This system can be solved using any method for solving linear problems. algebraic equations(for example, by the Gaussian method). As a result of the solution, unknown parameters of the approximating function will be found that provide the minimum sum of squared deviations of the approximating function from the original data, i.e. best possible quadratic approximation. It should be remembered that if even one value of the source data changes, all coefficients will change their values, since they are completely determined by the source data.

Approximation of source data by linear dependence

(linear regression)

As an example, let's consider the technique for determining the approximating function, which is specified in the form of a linear dependence. In accordance with the least squares method, the condition for the minimum of the sum of squared deviations is written in the following form:

Coordinates of table nodes;

Unknown coefficients of the approximating function, which is specified as a linear dependence.

A necessary condition for the existence of a minimum of a function is the equality to zero of its partial derivatives with respect to unknown variables. As a result, we obtain the following system of equations:

Let us transform the resulting linear system of equations.

We solve the resulting system of linear equations. The coefficients of the approximating function in analytical form are determined as follows (Cramer’s method):

These coefficients ensure the construction of a linear approximating function in accordance with the criterion of minimizing the sum of squares of the approximating function from the given tabular values ​​(experimental data).

Algorithm for implementing the least squares method

1. Initial data:

An array of experimental data with the number of measurements N is specified

The degree of the approximating polynomial (m) is specified

2. Calculation algorithm:

2.1. The coefficients for constructing a system of equations with dimensions are determined

Coefficients of the system of equations (left side of the equation)

- index of the column number of the square matrix of the system of equations

Free terms of a system of linear equations (right side of the equation)

- index of the row number of the square matrix of the system of equations

2.2. Formation of a system of linear equations with dimension .

2.3. Solving a system of linear equations to determine the unknown coefficients of an approximating polynomial of degree m.

2.4. Determination of the sum of squared deviations of the approximating polynomial from the original values ​​at all nodal points

The found value of the sum of squared deviations is the minimum possible.

Approximation using other functions

It should be noted that when approximating the original data in accordance with the least squares method, the logarithmic function, exponential function and power function are sometimes used as the approximating function.

Logarithmic approximation

Let's consider the case when the approximating function is given by a logarithmic function of the form:

The least squares method is one of the most common and most developed due to its simplicity and efficiency of methods for estimating parameters of linear. At the same time, when using it, some caution should be observed, since models constructed using it may not satisfy a number of requirements for the quality of their parameters and, as a result, do not reflect the patterns of process development “well” enough.

Let us consider the procedure for estimating the parameters of a linear econometric model using the least squares method in more detail. Such a model in general can be represented by equation (1.2):

y t = a 0 + a 1 x 1 t +...+ a n x nt + ε t.

The initial data when estimating the parameters a 0 , a 1 ,..., a n is a vector of values ​​of the dependent variable y= (y 1 , y 2 , ... , y T)" and the matrix of values ​​of independent variables

in which the first column, consisting of ones, corresponds to the model coefficient.

The least squares method received its name based on the basic principle that the parameter estimates obtained on its basis must satisfy: the sum of squares of the model error should be minimal.

Examples of solving problems using the least squares method

Example 2.1. The trading enterprise has a network of 12 stores, information on the activities of which is presented in table. 2.1.

The management of the enterprise would like to know how the annual amount depends on the retail space of the store.

Table 2.1

Least squares solution. Let us denote the annual turnover of the th store, million rubles; — retail area of ​​the th store, thousand m2.

Fig.2.1. Scatterplot for Example 2.1

To determine the form of the functional relationship between the variables and we will construct a scatter diagram (Fig. 2.1).

Based on the scatter diagram, we can conclude that annual turnover is positively dependent on retail space (i.e., y will increase with increasing ). The most suitable form of functional connection is linear.

Information for further calculations is presented in table. 2.2. Using the least squares method, we estimate the parameters of a linear one-factor econometric model

Table 2.2

Thus,

Therefore, with an increase in retail space by 1 thousand m2, other things being equal, the average annual turnover increases by 67.8871 million rubles.

Example 2.2. The company's management noticed that the annual turnover depends not only on the store's sales area (see example 2.1), but also on the average number of visitors. The relevant information is presented in table. 2.3.

Table 2.3

Solution. Let us denote the average number of visitors to the th store per day, thousand people.

To determine the form of the functional relationship between the variables and we will construct a scatter diagram (Fig. 2.2).

Based on the scatterplot, we can conclude that annual turnover is positively dependent on the average number of visitors per day (i.e., y will increase with increasing ). The form of functional dependence is linear.

Rice. 2.2. Scatterplot for Example 2.2

Table 2.4

In general, it is necessary to determine the parameters of a two-factor econometric model

y t = a 0 + a 1 x 1 t + a 2 x 2 t + ε t

The information required for further calculations is presented in table. 2.4.

Let us estimate the parameters of a linear two-factor econometric model using the least squares method.

Thus,

Estimation of the coefficient =61.6583 shows that, other things being equal, with an increase in retail space by 1 thousand m 2, the annual turnover will increase by an average of 61.6583 million rubles.

If a certain physical quantity depends on another quantity, then this dependence can be studied by measuring y at different values ​​of x. As a result of measurements, a number of values ​​are obtained:

x 1, x 2, ..., x i, ..., x n;

y 1 , y 2 , ..., y i , ... , y n .

Based on the data of such an experiment, it is possible to construct a graph of the dependence y = ƒ(x). The resulting curve makes it possible to judge the form of the function ƒ(x). However, the constant coefficients that enter into this function remain unknown. They can be determined using the least squares method. Experimental points, as a rule, do not lie exactly on the curve. The least squares method requires that the sum of the squares of the deviations of the experimental points from the curve, i.e.

2 was the smallest.

In practice, this method is most often (and most simply) used in the case of a linear relationship, i.e. When y = kx or

Linear dependence is very widespread in physics. And even when the relationship is nonlinear, they usually try to construct a graph so as to get a straight line. For example, if it is assumed that the refractive index of glass n is related to the light wavelength λ by the relation n = a + b/λ 2, then the dependence of n on λ -2 is plotted on the graph.

Consider the dependency In practice, this method is most often (and most simply) used in the case of a linear relationship, i.e. When(a straight line passing through the origin). Let's compose the value φ the sum of the squares of the deviations of our points from the straight line

The value of φ is always positive and turns out to be smaller the closer our points are to the straight line. The least squares method states that the value for k should be chosen such that φ has a minimum

or
(19)

The calculation shows that the root-mean-square error in determining the value of k is equal to

, (20)
where n is the number of measurements.

Let us now consider a slightly more difficult case, when the points must satisfy the formula y = a + bx(a straight line not passing through the origin).

The task is to find the best values ​​of a and b from the available set of values ​​x i, y i.

Let us again compose the quadratic form φ, equal to the sum of the squared deviations of points x i, y i from the straight line

and find the values ​​of a and b for which φ has a minimum

;

.

The joint solution of these equations gives

(21)

The root mean square errors of determination of a and b are equal

(23)

.  (24)

When processing measurement results using this method, it is convenient to summarize all the data in a table in which all the amounts included in formulas (19)(24) are preliminarily calculated. The forms of these tables are given in the examples below.

Example 1. The basic equation of the dynamics of rotational motion ε = M/J (a straight line passing through the origin) was studied. At different values ​​of the moment M, the angular acceleration ε of a certain body was measured. It is required to determine the moment of inertia of this body. The results of measurements of the moment of force and angular acceleration are listed in the second and third columns table 5.

Table 5

Using formula (19) we determine:

.

To determine the root mean square error, we use formula (20)

0.005775kg-1 · m -2 .

According to formula (18) we have

S J = (2.996 0.005775)/0.3337 = 0.05185 kg m2.

Having set the reliability P = 0.95, using the table of Student coefficients for n = 5, we find t = 2.78 and determine the absolute error ΔJ = 2.78 0.05185 = 0.1441 ≈ 0.2 kg m2.

Let's write the results in the form:

J = (3.0 ± 0.2) kg m2;

Example 2. Let's calculate the temperature coefficient of metal resistance using the least squares method. Resistance depends linearly on temperature

R t = R 0 (1 + α t°) = R 0 + R 0 α t°.

The free term determines the resistance R 0 at a temperature of 0 ° C, and the slope coefficient is the product of the temperature coefficient α and the resistance R 0 .

The results of measurements and calculations are given in the table ( see table 6).

Table 6

Using formulas (21), (22) we determine

R 0 = ¯ R- α R 0 ¯ t = 1.4005 - 0.002645 85.83333 = 1.1735 Ohm.

Let's find an error in the definition of α. Since , then according to formula (18) we have:

.

Using formulas (23), (24) we have

;

0.014126 Ohm.

Having set the reliability to P = 0.95, using the table of Student coefficients for n = 6, we find t = 2.57 and determine the absolute error Δα = 2.57 0.000132 = 0.000338 deg -1.

α = (23 ± 4) 10 -4 hail-1 at P = 0.95.

Example 3. It is required to determine the radius of curvature of the lens using Newton's rings. The radii of Newton's rings r m were measured and the numbers of these rings m were determined. The radii of Newton's rings are related to the radius of curvature of the lens R and the ring number by the equation

r 2 m = mλR - 2d 0 R,

where d 0 the thickness of the gap between the lens and the plane-parallel plate (or the deformation of the lens),

λ wavelength of incident light.

λ = (600 ± 6) nm;
r 2 m = y;
m = x;
λR = b;
-2d 0 R = a,

then the equation will take the form y = a + bx.

The results of measurements and calculations are entered into table 7.

Table 7

The task is to find the linear dependence coefficients at which the function of two variables A And b takes the smallest value. That is, given A And b the sum of squared deviations of the experimental data from the found straight line will be the smallest. This is the whole point of the least squares method.

Thus, solving the example comes down to finding the extremum of a function of two variables.

Deriving formulas for finding coefficients. A system of two equations with two unknowns is compiled and solved. Finding the partial derivatives of a function by variables A And b, we equate these derivatives to zero.

We solve the resulting system of equations using any method (for example, the substitution method or the Cramer method) and obtain formulas for finding the coefficients using the least squares method (LSM).

Given A And b function takes the smallest value.

That's the whole method of least squares. Formula for finding the parameter a contains the sums , , , and parameter n- amount of experimental data. We recommend calculating the values ​​of these amounts separately. Coefficient b found after calculation a.

The main area of ​​application of such polynomials is the processing of experimental data (construction of empirical formulas). The fact is that an interpolation polynomial constructed from function values ​​obtained through experiment will be strongly influenced by “experimental noise”; moreover, when interpolating, interpolation nodes cannot be repeated, i.e. The results of repeated experiments under the same conditions cannot be used. The root mean square polynomial smooths out noise and allows you to use the results of multiple experiments.

Numerical integration and differentiation. Example.

Numerical integration– calculation of the value of a definite integral (usually approximate). Numerical integration is understood as a set of numerical methods for finding the value of a certain integral.

Numerical differentiation– a set of methods for calculating the value of the derivative of a discretely specified function.

Integration

Formulation of the problem. Mathematical formulation of the problem: it is necessary to find the value of a definite integral

where a, b are finite, f(x) is continuous on [a, b].

When solving practical problems, it often happens that the integral is inconvenient or impossible to take analytically: it may not be expressed in elementary functions, the integrand can be given in the form of a table, etc. In such cases, numerical integration methods are used. Numerical integration methods use replacing the area of ​​a curvilinear trapezoid with a finite sum of areas of simpler ones geometric shapes, which can be calculated exactly. In this sense, they talk about using quadrature formulas.

Most methods use a representation of the integral as a finite sum (quadrature formula):

The basis of quadrature formulas is the idea of ​​replacing the graph of the integrand on the integration segment with functions of a simpler form, which can easily be integrated analytically and, thus, easily calculated. The task of constructing quadrature formulas is most easily implemented for polynomial mathematical models.

Three groups of methods can be distinguished:

1. Method with dividing the integration segment into equal intervals. Partitioning into intervals is done in advance; usually the intervals are chosen equal (to make it easier to calculate the function at the ends of the intervals). Calculate areas and sum them up (rectangle, trapezoid, Simpson methods).

2. Methods with partitioning the integration segment using special points (Gauss method).

3. Calculation of integrals using random numbers (Monte Carlo method).

Rectangle method. Let the function (figure) need to be integrated numerically on the segment . Divide the segment into N equal intervals. The area of ​​each of N curved trapezoids can be replaced by the area of ​​a rectangle.

The width of all rectangles is the same and equals:

To select the height of the rectangles, you can select the value of the function on the left border. In this case, the height of the first rectangle will be f(a), the second - f(x 1),..., N-f(N-1).

If we take the value of the function on the right border to select the height of the rectangle, then in this case the height of the first rectangle will be f(x 1), the second - f(x 2), ..., N - f(x N).

As you can see, in this case one of the formulas gives an approximation to the integral with an excess, and the second with a deficiency. There is another way - to use the value of the function in the middle of the integration segment for approximation:

Estimation of the absolute error of the rectangle method (middle)

Estimation of the absolute error of the left and right rectangle methods.

Example. Calculate for the entire interval and dividing the interval into four sections

Solution. Analytical calculation of this integral gives I=arctg(1)–arctg(0)=0.7853981634. In our case:

1)h = 1; xo = 0; x1 = 1;

2) h = 0.25 (1/4); x0 = 0; x1 = 0.25; x2 = 0.5; x3 = 0.75; x4 = 1;

Let's calculate using the left rectangle method:

Let's calculate using the right rectangle method:

Let's calculate using the average rectangle method:

Trapezoid method. Using a first-degree polynomial (a straight line drawn through two points) to interpolate results in the trapezoidal formula. The ends of the integration segment are taken as interpolation nodes. Thus, the curvilinear trapezoid is replaced by an ordinary trapezoid, the area of ​​which can be found as the product of half the sum of the bases and the height

In the case of N integration segments for all nodes, with the exception of extreme points segment, the value of the function will be included in the total sum twice (since adjacent trapezoids have one common side)

The trapezoid formula can be obtained by taking half the sum of the formulas of rectangles along the right and left edges of the segment:

Checking the stability of the solution. As a rule, the shorter the length of each interval, i.e. the greater the number of these intervals, the less the difference between the approximate and exact values ​​of the integral. This is true for most functions. In the trapezoid method, the error in calculating the integral ϭ is approximately proportional to the square of the integration step (ϭ ~ h 2). Thus, to calculate the integral of a certain function in terms of a, b, it is necessary to divide the segment into N 0 intervals and find the sum of the areas of the trapezoid. Then you need to increase the number of intervals N 1, again calculate the sum of the trapezoid and compare the resulting value with the previous result. This should be repeated until (N i) until the specified accuracy of the result is achieved (convergence criterion).

For the rectangle and trapezoid methods, usually at each iteration step the number of intervals increases by 2 times (N i +1 = 2N i).

Convergence criterion:

The main advantage of the trapezoidal rule is its simplicity. However, if high precision is required when calculating the integral, this method may require too many iterations.

Absolute error of the trapezoidal method is estimated as
.

Example. Calculate an approximately definite integral using the trapezoidal formula.

a) Dividing the segment of integration into 3 parts.
b) Dividing the segment of integration into 5 parts.

Solution:
a) According to the condition, the segment of integration must be divided into 3 parts, that is.
Let's calculate the length of each partition segment: .

Thus, the general formula for trapezoids is reduced to a nice size:

Finally:

Let me remind you that the obtained value is an approximate value of the area.

b) Let's divide the integration segment into 5 equal parts, that is. By increasing the number of segments, we increase the accuracy of calculations.

If , then the trapezoidal formula takes the following form:

Let's find the partition step:
, that is, the length of each intermediate segment is 0.6.

When finalizing the task, all calculations are conveniently formatted calculation table:

In the first line we write “counter”

As a result:

Well, there really is a clarification, and a serious one!
If for 3 partition segments, then for 5 segments. If you take an even larger segment => it will be even more accurate.

Simpson's formula. The trapezoidal formula gives a result that strongly depends on the step size h, which affects the accuracy of calculating a certain integral, especially in cases where the function is non-monotonic. We can assume an increase in the accuracy of calculations if, instead of straight segments replacing curvilinear fragments of the graph of the function f(x), we use, for example, fragments of parabolas given through three adjacent points of the graph. This geometric interpretation underlies Simpson's method for calculating the definite integral. The entire integration interval a,b is divided into N segments, the length of the segment will also be equal to h=(b-a)/N.

Simpson's formula looks like:

remainder term

As the length of the segments increases, the accuracy of the formula decreases, so to increase the accuracy, Simpson's compound formula is used. The entire integration interval is divided into an even number of identical segments N, the length of the segment will also be equal to h=(b-a)/N. Simpson's compound formula is:

In the formula, the expressions in brackets represent the sums of the values ​​of the integrand at the ends of the odd and even internal segments, respectively.

The remainder of Simpson's formula is proportional to the fourth power of the step:

Example: Using Simpson's rule, calculate the integral. (Exact solution - 0.2)

Gauss method

Gaussian quadrature formula. The basic principle of quadrature formulas of the second type is visible from Figure 1.12: it is necessary to place the points in this way X 0 and X 1 inside the segment [ a;b], so that the total area of ​​the “triangles” is equal to the area of ​​the “segment”. When using the Gauss formula, the original segment [ a;b] is reduced to the segment [-1;1] by replacing the variable X on

0.5∙(b– a)∙t+ 0.5∙(b + a).

Then , Where .

Such a replacement is possible if a And b are finite, and the function f(x) is continuous on [ a;b]. Gauss formula at n points x i, i=0,1,..,n-1 inside the segment [ a;b]:

, (1.27)

Where t i And A i for various n are given in reference books. For example, when n=2 A 0 =A 1 =1; at n=3: t 0 =t 2 "0.775, t 1 =0, A 0 =A 2 "0.555, A 1 "0.889.

Gaussian quadrature formula

obtained with a weight function equal to unity p(x)= 1 and nodes x i, which are the roots of the Legendre polynomials

Odds A i easy to calculate using formulas

i=0,1,2,...n.

The values ​​of nodes and coefficients for n=2,3,4,5 are given in the table

Example. Calculate the value using the Gauss formula for n=2:

Exact value: .

The algorithm for calculating the integral using the Gauss formula does not involve doubling the number of microsegments, but increasing the number of ordinates by 1 and comparing the obtained values ​​of the integral. The advantage of the Gauss formula is its high accuracy with a relatively small number of ordinates. Disadvantages: inconvenient for manual calculations; it is necessary to store the values ​​in the computer memory t i, A i for various n.

The error of the Gaussian quadrature formula on the segment will be For the remainder term formula will be and the coefficient α N decreases quickly with growth N. Here

Gaussian formulas provide high accuracy even with a small number of nodes (from 4 to 10). In this case, in practical calculations the number of nodes ranges from several hundred to several thousand. Note also that the weights of Gaussian quadratures are always positive, which ensures the stability of the algorithm for calculating the sums