Squares. Method of Least

FIGURE 6.12 An example of a standard curve in which, due to random errors, the data points do not lie on the line. 

See the following reference for details Harvey, David, Modern Analytical Chemistry, McGraw-Hill (2000), page 119. 

Below is illustrated the method of least squares to fit a straight line to a set of data points (j Xj). Extensions to nonlinear least squares fits are discussed in Section B.4. 

Consider the problem of fitting a set of data (y x,) where y and x are the dependent and independent variables, respectively, to an equation of the form  

For any value of x = x the probability PPj for making the observed measurement y, with a Gaussian distribution and a standard deviation ct,- for the observations about the actual value y(x,-) is (P. R. Bevington, Data Reduction and Error Analysis for the Physical Sciences, McGraw-Hill, New York, 1969, p. 101)  

The probability of making the observed data set of measurements of the N values of y, is the product of the individual PF, or  

The best estimates for aj and 2 are the values that maximize PP(a, as) (method of maximum likelihood). Define  

If a number of measurements are made of the same physical quantity and if these measurements are subject only to random errors, then the theory of least squares states that the most probable value of the measured quantity is that which makes the sum of the squares of the errors a minimum. 

This theorem is applied as follows to the problem of finding the straight line which best fits a set of experimentally determined points. If there are only two points, there is no problem, because the two constants which define a straight line can be unequivocally determined from these two points. But, in general, there will be more points available than constants to be determined. Suppose that the various points have coordinates X2y2, 373 and that it is known that x and y are related by an equation of the form 

Our problem is to find the values of the constants a and b, since these define the straight line. In general, the line will not pass exactly through any of the points since each is subject to a random error. Therefore each point is in error by an amount given by its deviation from the straight line. For example, Eq. (11-14) states that the value of y corresponding to x = x, is a + bx ). Yet the first experimental point has a value of y =. Therefore e, the error in the first point, 

We can calculate the errors in the other points in similar fashion, and then write down the expression for the sum of the squares of these errors  

According to the theory of least squares, the best straight line is that which makes the sum of the squared errors a minimum. Therefore, the best value of a is found by differentiating Eq. (11-15) with respect to a and equating the result to zero  

---

This method, because it involves minimizing the sum of squares of the deviations xi — p, is called the method of least squares. We have encountered the principle before in our discussion of the most probable velocity of an individual particle (atom or molecule), given a Gaussian distr ibution of particle velocities. It is ver y powerful, and we shall use it in a number of different settings to obtain the best approximation to a data set of scalars (arithmetic mean), the best approximation to a straight line, and the best approximation to parabolic and higher-order data sets of two or more dimensions. 

If the experimental error is random, the method of least squares applies to analysis of the set. Minimize the sum of squares of the deviations by differentiating with respect to m. 

Linear regression, also known as the method of least squares, is covered in Section 5C. 

Application. Merriman ( The Method of Least Squares Applied to a Hydraulic Problem, y, Franklin Inst., 23.3-241, October 1877) reported on a study of stream velocity as a function of relative depth of the stream. 

A straight line may be fitted to the (X, Y) or (X, Y) pairs of data when plotted on log-log graph paper from which the slope N and the intercept log K with X = 1 may be read. Alternatively, the method of least squares may be used to estimate the values of K and N, giving the best fit to the available data. 

Nonlinear regression, a technique that fits a specified function of x and y by the method of least squares (i.e., the sum of the squares of the differences between real data points and calculated data points is minimized). 

Once a linear relationship has been shown to have a high probability by the value of the correlation coefficient (r), then the best straight line through the data points has to be estimated. This can often be done by visual inspection of the calibration graph but in many cases it is far better practice to evaluate the best straight line by linear regression (the method of least squares). 

This approach was applied to data obtained by Hausberger, Atwood, and Knight (17). Figure 9 shows the basic temperature profile and feed gas data and the derived composition profiles. Application of the Hougen and Watson approach (16) and the method of least squares to the calculated profiles in Figure 9 gave the following methane rate equation ... 

The method of least squares provides the most powerful and useful procedure for fitting data. Among other applications in kinetics, least squares is used to calculate rate constants from concentration-time data and to calculate other rate constants from the set of -concentration values, such as those depicted in Fig. 2-8. If the function is linear in the parameters, the application is called linear least-squares regression. The more general but more complicated method is nonlinear least-squares regression. These are examples of linear and nonlinear equations ... 

The isotherms can be approximated by the well-known method of least squares, in which the function F is to be minimized ... 

The quantities boo and Soo represent the solution of the problem of drawing parallel lines through the given set of points by the method of least squares. They are obtained relatively easily and can serve to check the whole calculation. The function Sx = f(x) always has a minimum however, the conditions have not been explored when it has only one minimum. This case would be important from the theoretical point of view however, in practice there is no real danger... 

Huyberechts, S., A. Halleux, and P, Kruys,Bu//. Soc. Chim. Beiges, 64, 203 (1955). Linnik, Yu. V., Me tod Naimenshikh Kvadratov i Osnovy Matematichesko-Statisticheskoi Teorii Obrabotki Nablyudenii (Method of Least Squares and Principles of Mathematico-Statistical Theory of Data Processing), Gos. Izdatelstvo Fiz. Mat. Literatury, Moscow, 1958. 

Deutsch and Hansch applied this principle to the sweet taste of the 2-substituted 5-nitroanilines. Using the data available (see Table VII), the calculated regression Eqs. 5-7 (using the method of least squares) optimally expressed the relationship between relative sweetness (RS), the Hammett constant, cr, and the hydrophobic-bonding constant, ir. 

Table 2.3 is used to classify the differing systems of equations, encountered in chemical reactor applications and the normal method of parameter identification. As shown, the optimal values of the system parameters can be estimated using a suitable error criterion, such as the methods of least squares, maximum likelihood or probability density function. 

Gans, P., Data Fitting in the Chemical Sciences by the Method of Least Squares, Wiley, New York, NY, (1992). 

Plackett, R.L., "Studies in the History of Probability and Statistic. XXIX The Discovery of the method of least squares", Biometrika, 59 (2), 239-251 (1972). 

Another advantage of the Savitsky-Golay method is that derivatives of these functions can also be determined from the method of least squares. This method can be used to determine alpha-peak temperatures automatically since the first derivative changes sign at the peak temperature. The advantage of smoothing is that the number of extraneous peaks due to noise has been minimized. 

Data may be fitted to this equation by the method of least squares in order to determine values of the constants log /c, fiA, / B, etc. The goodness of fit may be shown in graphical form by using the values of determined in this manner to calculate (Cf). A plot of the reaction rate versus this function should then meet the criteria of the general method outlined above. 

What is the method of least squares and why is it useful in instrumental analysis ... 

What is it about the calculations involved in the method of least squares that gives this method its name ... 

Give four parameters that are readily obtainable as a result of the method of least squares treatment of a set of data. 

The method of least squares is a procedure by which the best straight line through a series of data points is mathematically determined. More details are given in Section 6.4.4. It is useful because it eliminates guesswork as to the exact placement of the line and provides the slope and y-intercept of the line. 

Linear regression is another name for the process of determining the straight line for a series of data points via the method of least squares. 

---